ION Capability Container and core framework (python) - an advanced fork is available at https://github.com/scionrep/scioncc

ooici, updated 🕥 2022-07-06 19:14:58

=========================================================== Ocean Observatories Initiative Cyberinfrastructure (OOI CI) Integrated Observatory Network (ION) - OOINet

pyon - Python Capability Container and Core Modules https://confluence.oceanobservatories.org/x/ehdTAQ

(C) The Regents of the University of California, 2010-2014 Open source under BSD license. See LICENSE.txt for details. ===========================================================

INSTALL

Install the following if not yet present:

OS Packages and package management: For Mac, use homebrew > /usr/bin/ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)
python 2.7.x > brew install python --framework --universal Note: see 'brew doctor' for any path issues
git 1.7.7 or higher: Use brew or download the Mac or Linux installer and run it > brew install git
Install core packages with Homebrew

brew install libevent libyaml zeromq rabbitmq hdf5 pkg-config netcdf udunits

Linux: Note that many installs have much older versions installed by default. (If you have trouble with brew 'MD5 mismatch' errors, try running 'brew update' and try the install again) For any kind of installation trouble run 'brew doctor' and fix any issues.
Install Postgres with Homebrew

brew install postgres postgis

If you have trouble with the installation, please see: https://confluence.oceanobservatories.org/display/CIDev/Postgres+Datastore https://confluence.oceanobservatories.org/display/CIDev/M112+Geospatial+Search

As user postgres:

createuser -s ion
Python packages and environment management:
pip > easy_install pip
virtualenv and virtualenvwrapper modules for your python 2.7 installation > easy_install --upgrade virtualenv > easy_install --upgrade virtualenvwrapper Note: This require Mac's XCode with installed command line tools
Setup a virtualenv to use for Pyon: > mkvirtualenv --no-site-packages pyon
Pin setuptools in your virtualenv > pip install -U setuptools==0.8
Install numpy to the virtualenv > pip install numpy==1.7.1

DEVELOPMENT TIME DEPENDENCIES

PACKAGES:

Use Buildout to install the python package dependencies with: > python bootstrap.py -v 2.2.0 > bin/buildout

** Add following to your .bashrc profile on Macs if buildout fails to compile for packages such as gevent, pyml, etc. export C_INCLUDE_PATH=/usr/local/include

ION-DEFINITIONS SUBMODULE:

Link git submodule ion-definitions, containing object and service definitions and resource files: git submodule update --init

Note: This git-submodule always references a specific commit of ion-definitions used by pyon.

Hint: The .gitmodules file declares the submodule as read only GIT reference (https: instead of [email protected]). If you have problems with a read only (anonymous) clone because of a missing public key, edit .git/config and change the last url to git://github.com/ooici/ion-definitions.git

If you want to avoid entering your password for every push to the submodule,
edit extern/ion-definitions/.git/config and change the remote reference to:
[email protected]:ooici/ion-definitions.git

To update the code in pyon and the specific commit in the submodule: > git pull > git submodule update

Notes: - The submodule update does NOT happen automatically - The submodule update does NOT pull the branch head of the submodule. - To change the submodule commit within pyon, cd into the submodule and do a git pull, then commit pyon.

The obj/ subdir is a symlink to the checkout's extern/ion-definitions/objects. The res/ subdir is a symlink to the checkout's extern/ion-definitions/res.

See more about submodules: - http://book.git-scm.com/5_submodules.html - http://speirs.org/blog/2009/5/11/understanding-git-submodules.html - http://blog.endpoint.com/2010/04/git-submodule-workflow.html

GENERATE SERVICE STUBS:

Generate service definition interfaces and abstract base classes files: > bin/generate_interfaces Note: These will be placed in the interfaces/ subdirectory (and not in GIT nor in the egg).

RUN-TIME DEPENDENCIES

POSTGRESQL (DATABASE):

Run postgresql in second terminal before starting the container or running tests: > /usr/local/bin/postgres -D /usr/local/var/postgres

RABBITMQ (MESSAGE BROKER): > rabbitmq-server

SOURCE PATH:

Add the current directory to your PYTHONPATH for an easier time running scripts. Add the following line to your ~/.profile (create it if it doesn't exist, or use bash_profile if you already have that): export PYTHONPATH=.

TEST

Run the unittests: bin/nosetests -a UNIT

Run code coverage: bin/nosetests -w pyon --with-coverage --cover-package=pyon

USE

Provide local configuration overrides in: res/config/pyon.local.yml res/config/logging.local.yml

Start the capability container in interactive mode: > bin/pycc

List supported arguments: > bin/pycc --help

Start the CC with a deploy file: > bin/pycc --rel res/deploy/r2deploy.yml

Start the CC with a process to execute and then quit: > bin/pycc --proc examples.service.hello_service.HelloService

Command line configuration options for pycc: Use a different message broker: > bin/pycc server.amqp.host=amoeba.ucsd.edu

Use a different database server:
> bin/pycc server.couchdb.host=amoeba.ucsd.edu

Set the container's system name:
> bin/pycc system.name=MINE

DEBUG

USE Pycharm or any other Python IDE to debug; breakpoints and step-by-step are possible. Set the executable to the bin/pycc and set the working directory to the repository root. Use the -n option to avoid the interactive ipython shell. It does not work property with the debugger.

You will need to make sure the Pycharm debug egg is available in your path, something like: > export PYTHONPATH=.:/Applications/PyCharm\ 1.5.2.app/pycharm-debug.egg

Remote debugging of services in a running container is possible too. Setup a remote debugging configuration to use the localhost and port 8585. First start the remote debuging server, then start the container from the command line below (the -n seems to be required) > bin/pyccd -n --rel res/deploy/r2services.yml

DATASTORE

Use psql

EXAMPLES

Make sure to look in the examples directory for a README and examples.

https://confluence.oceanobservatories.org/display/CIDev/ION+R2+Development https://confluence.oceanobservatories.org/display/CIDev/R2+Container+Use+Guide https://confluence.oceanobservatories.org/display/CIDev/R2+Development+Conventions https://confluence.oceanobservatories.org/display/CIDev/R2+Resource+Development+Guide https://confluence.oceanobservatories.org/display/CIDev/R2+Service+Implementation+Guide https://confluence.oceanobservatories.org/display/CIDev/R2+Service+Testing+Guide

https://confluence.oceanobservatories.org/x/ehdTAQ https://github.com/ooici/coi-services

Issues

Bump lxml from 2.3.4 to 4.9.1

opened on 2022-07-06 19:14:55 by dependabot[bot]

Bumps lxml from 2.3.4 to 4.9.1.

Changelog

Sourced from lxml's changelog.

4.9.1 (2022-07-01)

Bugs fixed

A crash was resolved when using iterwalk() (or canonicalize()) after parsing certain incorrect input. Note that iterwalk() can crash on valid input parsed with the same parser after failing to parse the incorrect input.

4.9.0 (2022-06-01)

Bugs fixed

GH#341: The mixin inheritance order in lxml.html was corrected. Patch by xmo-odoo.

Other changes

Built with Cython 0.29.30 to adapt to changes in Python 3.11 and 3.12.

Wheels include zlib 1.2.12, libxml2 2.9.14 and libxslt 1.1.35 (libxml2 2.9.12+ and libxslt 1.1.34 on Windows).

GH#343: Windows-AArch64 build support in Visual Studio. Patch by Steve Dower.

4.8.0 (2022-02-17)

Features added

GH#337: Path-like objects are now supported throughout the API instead of just strings. Patch by Henning Janssen.

The ElementMaker now supports QName values as tags, which always override the default namespace of the factory.

Bugs fixed

GH#338: In lxml.objectify, the XSI float annotation "nan" and "inf" were spelled in lower case, whereas XML Schema datatypes define them as "NaN" and "INF" respectively.

... (truncated)

Commits

d01872c Prevent parse failure in new test from leaking into later test runs.
d65e632 Prepare release of lxml 4.9.1.
86368e9 Fix a crash when incorrect parser input occurs together with usages of iterwa...
50c2764 Delete unused Travis CI config and reference in docs (GH-345)
8f0bf2d Try to speed up the musllinux AArch64 build by splitting the different CPytho...
b9f7074 Remove debug print from test.
b224e0f Try to install 'xz' in wheel builds, if available, since it's now needed to e...
897ebfa Update macOS deployment target version from 10.14 to 10.15 since 10.14 starts...
853c9e9 Prepare release of 4.9.0.
d3f77e6 Add a test for https://bugs.launchpad.net/lxml/+bug/1965070 leaving out the a...
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/ooici/pyon/network/alerts).

Bump ipython from 0.13.0 to 7.16.3

opened on 2022-01-21 18:56:13 by dependabot[bot]

Bumps ipython from 0.13.0 to 7.16.3.

Release notes

Sourced from ipython's releases.

7.9.0

No release notes provided.

7.8.0

No release notes provided.

7.7.0

No release notes provided.

7.6.1

No release notes provided.

7.6.0

No release notes provided.

7.5.0

No release notes provided.

7.4.0

No release notes provided.

7.3.0

No release notes provided.

7.2.0

No release notes provided.

7.1.1

No release notes provided.

7.1.0

No release notes provided.

7.0.1

No release notes provided.

7.0.0

No release notes provided.

7.0.0-doc

No release notes provided.

7.0.0rc1

No release notes provided.

7.0.0b1

No release notes provided.

6.2.1

No release notes provided.

... (truncated)

Commits

d43c7c7 release 7.16.3
5fa1e40 Merge pull request from GHSA-pq7m-3gw7-gq5x
8df8971 back to dev
9f477b7 release 7.16.2
138f266 bring back release helper from master branch
5aa3634 Merge pull request #13341 from meeseeksmachine/auto-backport-of-pr-13335-on-7...
bcae8e0 Backport PR #13335: What's new 7.16.2
8fcdcd3 Pin Jedi to <0.17.2.
2486838 release 7.16.1
20bdc6f fix conda build
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump psutil from 1.0.1 to 5.6.6

opened on 2020-03-12 17:03:32 by dependabot[bot]

Bumps psutil from 1.0.1 to 5.6.6.

Changelog

Sourced from psutil's changelog.

5.6.6

2019-11-25

Bug fixes

1179_: [Linux] Process cmdline() now takes into account misbehaving processes renaming the command line and using inappropriate chars to separate args.

1616_: use of Py_DECREF instead of Py_CLEAR will result in double free and segfault (CVE-2019-18874 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-18874>__). (patch by Riccardo Schirone)

1619_: [OpenBSD] compilation fails due to C syntax error. (patch by Nathan Houghton)

5.6.5

2019-11-06

Bug fixes

1615_: remove pyproject.toml as it was causing installation issues.

5.6.4

2019-11-04

Enhancements

1527_: [Linux] added Process.cpu_times().iowait counter, which is the time spent waiting for blocking I/O to complete.

1565_: add PEP 517/8 build backend and requirements specification for better pip integration. (patch by Bernát Gábor)

Bug fixes

875_: [Windows] Process' cmdline(), environ() or cwd() may occasionally fail with ERROR_PARTIAL_COPY which now gets translated to AccessDenied.

1126_: [Linux] cpu_affinity() segfaults on CentOS 5 / manylinux. cpu_affinity() support for CentOS 5 was removed.

1528_: [AIX] compilation error on AIX 7.2 due to 32 vs 64 bit differences. (patch by Arnon Yaari)

1535_: 'type' and 'family' fields returned by net_connections() are not always turned into enums.

1536_: [NetBSD] process cmdline() erroneously raise ZombieProcess error if cmdline has non encodable chars.

1546_: usage percent may be rounded to 0 on Python 2.

... (truncated)

Commits

c6cd256 pre release
b2414b8 revert #1595
c63369e updat HISTORY
edb20f6 linux, cmdline(), fix for #1179, comment 552984549: sometimes string ends wit...
d739cbb use PROCESS_QUERY_LIMITED_INFORMATION
f7e898b #1595: use psutil_pid_is_running() instead of GetExitCodeProcess
72c84cb #fix #1595 / windows: kill() may not raise AccessDenied
1f8d432 Merge branch 'master' of github.com:giampaolo/psutil
e6faebc release gil around users()/BSD (#1425)
5cb1b0b Merge branch 'master' of github.com:giampaolo/psutil
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

M153: Multiple Broker Support

opened on 2014-06-11 20:29:26 by daf

Several underlying changes to pyon.net/pyon.ion.exchange infrastructure to more broadly support ExchangeObjects. Endpoints automatically upgrade to using ExchangeObjects when the ability becomes available. Events also updated.

PR for review / letting buildbot do it's thing. I've tested rather extensively locally but as this is a very substantial, sweeping change, there are likely going to be ripples.

Ocean Observatories Initiative CI

GitHub Repository Homepage

ION Capability Container and core framework (python) - an advanced fork is available at https://github.com/scionrep/scioncc

INSTALL

DEVELOPMENT TIME DEPENDENCIES

RUN-TIME DEPENDENCIES

TEST

USE

DEBUG

DATASTORE

EXAMPLES

SEE ALSO

Issues

Bump lxml from 2.3.4 to 4.9.1

4.9.1 (2022-07-01)

Bugs fixed

4.9.0 (2022-06-01)

Bugs fixed

Other changes

4.8.0 (2022-02-17)

Features added

Bugs fixed

Bump ipython from 0.13.0 to 7.16.3

7.9.0

7.8.0

7.7.0

7.6.1

7.6.0

7.5.0

7.4.0

7.3.0

7.2.0

7.1.1

7.1.0

7.0.1

7.0.0

7.0.0-doc

7.0.0rc1

7.0.0b1

6.2.1

Bump psutil from 1.0.1 to 5.6.6

5.6.6

5.6.5

5.6.4

M153: Multiple Broker Support

Ocean Observatories Initiative CI