Jupyter Notebook Extension for monitoring your own Resource Usage

jupyter-server, updated 🕥 2023-03-27 20:47:32

Installation | Configuration | Resources Displayed | Contributing

jupyter-resource-usage

Screenshot with memory limit

Jupyter Resource Usage is an extension for Jupyter Notebooks and JupyterLab that displays an indication of how much resources your current notebook server and its children (kernels, terminals, etc) are using. This is displayed in the main toolbar in the notebook itself, refreshing every 5s.

Kernel resource usage can be displayed in a sidebar for IPython kernels with ipykernel >= 6.11.0.

Installation

You can currently install this package from PyPI.

bash pip install jupyter-resource-usage

Or with conda:

bash conda install -c conda-forge jupyter-resource-usage

If your notebook version is < 5.3, you need to enable the extension manually.

jupyter serverextension enable --py jupyter_resource_usage --sys-prefix jupyter nbextension install --py jupyter_resource_usage --sys-prefix jupyter nbextension enable --py jupyter_resource_usage --sys-prefix

Configuration

Memory Limit

jupyter-resource-usage can display a memory limit (but not enforce it). You can set this in several ways:

MEM_LIMIT environment variable. This is set by JupyterHub if using a spawner that supports it.
In the commandline when starting jupyter notebook, as --ResourceUseDisplay.mem_limit.
In your Jupyter notebook traitlets config file

The limit needs to be set as an integer in Bytes.

Memory usage warning threshold

Screenshot with memory warning

The background of the resource display can be changed to red when the user is near a memory limit. The threshold for this warning can be configured as a fraction of the memory limit.

If you want to flash the warning to the user when they are within 10% of the memory limit, you can set the parameter --ResourceUseDisplay.mem_warning_threshold=0.1.

CPU Usage

jupyter-resource-usage can also track CPU usage and report a cpu_percent value as part of the /api/metrics/v1 response.

You can set the cpu_limit in several ways:

CPU_LIMIT environment variable. This is set by JupyterHub if using a spawner that supports it.
In the command line when starting jupyter notebook, as --ResourceUseDisplay.cpu_limit.
In your Jupyter notebook traitlets config file

The limit corresponds to the number of cpus the user has access to, but does not enforce it.

Additionally, you can set the track_cpu_percent trait to enable CPU usage tracking (disabled by default):

python c = get_config() c.ResourceUseDisplay.track_cpu_percent = True

As a command line argument:

bash jupyter notebook --ResourceUseDisplay.track_cpu_percent=True

Disable Prometheus Metrics

There is a known bug with Prometheus metrics which causes "lag"/pauses in the UI. To workaround this you can disable Prometheus metric reporting using:

--ResourceUseDisplay.enable_prometheus_metrics=False

Resources Displayed

Currently the server extension only reports memory usage and CPU usage. Other metrics will be added in the future as needed.

Memory usage will show the PSS whenever possible (Linux only feature), and default to RSS otherwise.

The notebook extension currently doesn't show CPU usage, only memory usage.

Contributing

If you would like to contribute to the project, please read the CONTRIBUTING.md. The CONTRIBUTING.md file explains how to set up a development installation and how to run the test suite.

Issues

chore: move `prettier` and `eslint` configuration under `package.json`

opened on 2023-03-27 20:47:32 by SauravMaheshkar

Changes proposed by this PR can be summarised as follows :-

Move the content from .prettierrc under the prettier key in package.json. Source(prettier docs)
Move the content from .eslintrc.js under the eslintConfig key in package.json. Source(eslint docs)
Move the content from .eslintignore under the eslintIgnore key in package.json. Source(eslint docs)

Doing so, would lead to a more minimal project structure without any functional change. This PR mirrors the change from (https://github.com/jupyter-server/jupyter_server/pull/1246)

Not working (no update of memory usage) after updating to JupyterLab 3.6.2

opened on 2023-03-24 00:27:42 by adalseno

Description

The info about RAM usage is not updated since I installed JupyterLab 3.6.2 Screenshot 2023-03-24 01 18 49 but the memory usage is much higher Screenshot 2023-03-24 01 22 31 With JupyerLab 3.6.1 it was working fine I have the same issue when running a notebook with R kernel (I don't remember if with 3.6.1 it was working; I usually use python kernels).

Reproduce

Go to '...'
Click on '...'
Scroll down to '...'
See error '...'

Expected behavior

RAM usage should update

Context

jupyter-resource-usage 0.7.2 installed via pip

Operating System and version:
Browser and version: Chrome Version 110.0.5481.181
Jupyter Server version: JupyterLab 3.6.2

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here, if applicable.

Reporting Disk Usage

opened on 2023-03-23 20:07:10 by laufers

Problem

I would like it, similar to cpu and memory usage, we could display disk usage to alert users that the local storage (/home/user) is approaching capacity.

Proposed Solution

Display disk usage similar to cpu and memory

Request for 1.0.0 (not 0.8.0) as the next non-bugfix release

opened on 2023-03-21 09:45:35 by consideRatio

I've took to heart a recommendation from https://semver.org suggesting one should quickly arrive at 1.0.0. Here is excerpt from the section of How do I know when to release 1.0.0?.

If your software is being used in production, it should probably already be 1.0.0. If you have a stable API on which users have come to depend, you should be 1.0.0. If you’re worrying a lot about backwards compatibility, you should probably already be 1.0.0.

What do you think about cutting 1.0.0 instead of 0.8.0 when something beyond a bugfix is made, or possibly instead of 0.7.3 as well with a very undramatic release with the purpose of just arriving to 1.0.0 to help communicate about future versions?

Further reasoning

I've grown very fond of both maintaining packages and being a user of packages maintained with a major version released. Like that as a maintainer, communication to users becomes easier just by the version choice.

This project is already helping communicate about changes via a CHANGELOG.md etc, and its also not a new project, so I think it makes perfect sense to make it 1.0.0.

I arrive here now as I saw a pinning in the jupyterhub/the-littlest-jupyterhub distribution of jupyterhub involving this:

nbgitpuller==1.* jupyter-resource-usage==0.6.*

We want to pin to the major version, but for jupyter-resource-usage we pin to the first version identifier instead as we can't expect there isn't any breaking changes between 0.6 and 0.7. Of course, 0.7 may simply have been a feature rich release that motivated a version bump from 0.6 to 0.7, and not something associated with breaking changes.

Getting 'Unhandled error' when it tries to open control channel

opened on 2023-02-24 22:37:35 by rucshah

Hi, Thanks for this great extension. I tried to configure this in my setup (JupyterServer + JupyterLab + Jupyter Enterprise Gateway) which is running in kubernetes. My setup has

bash jupyter_client 8.0.3 jupyter_core 5.2.0 jupyter-resource-usage 0.7.2 jupyter-server 1.23.42 jupyterhub 3.1.1 jupyterlab 3.5.2 jupyter_enterprise_gateway 3.1.0 ipykernel 6.21.1

But when extension makes a call, I see following exception in jupyter server: ```bash Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/tornado/web.py", line 1713, in _execute result = await result File "/opt/conda/lib/python3.10/site-packages/jupyter_resource_usage/api.py", line 115, in get control_channel = client.control_channel File "/opt/conda/lib/python3.10/site-packages/jupyter_server/gateway/managers.py", line 765, in control_channel assert self.channel_socket is not None AssertionError

[W 2023-02-24 22:25:02.920 SingleUserNotebookApp handlers:649] wrote error: 'Unhandled error' ```

Am I missing something here?