web scraping nike products

yassine-youcefi, updated 🕥 2022-12-08 07:56:43

web-scraping-nike-website

web scraping nike products from nike and save data in file.csv and file.json :

csv results

json results


Note:

To avoid the problems of web trackers, just active Blocking social media tracker utile of firefox

for use the script :

  1. Enter a folder that contains scrap.py with terminal
  2. python scrap.py

for use the api:

  • run api server run.py

to using neo.py :

  • pip install virtualenv
  • source venv/bin/activate
  • pip3 install -r requirements.txt


for scraping mor products add url to list my_url

Issues

Bump certifi from 2020.12.5 to 2022.12.7

opened on 2022-12-08 07:56:43 by dependabot[bot]

Bumps certifi from 2020.12.5 to 2022.12.7.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/yassine-youcefi/web-scraping-nike-website/network/alerts).

CVE-2007-4559 Patch

opened on 2022-11-23 00:59:00 by TrellixVulnTeam

Patching CVE-2007-4559

Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

Bump numpy from 1.19.5 to 1.22.0

opened on 2022-06-22 04:05:39 by dependabot[bot]

Bumps numpy from 1.19.5 to 1.22.0.

Release notes

Sourced from numpy's releases.

v1.22.0

NumPy 1.22.0 Release Notes

NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

  • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
  • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
  • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
  • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
  • A new configurable allocator for use by downstream projects.

These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

Expired deprecations

Deprecated numeric style dtype strings have been removed

Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

(gh-19539)

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

(gh-19615)

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/yassine-youcefi/web-scraping-nike-website/network/alerts).

Bump pyjwt from 1.7.1 to 2.4.0

opened on 2022-05-25 05:28:16 by dependabot[bot]

Bumps pyjwt from 1.7.1 to 2.4.0.

Release notes

Sourced from pyjwt's releases.

2.4.0

Security

What's Changed

New Contributors

Full Changelog: https://github.com/jpadilla/pyjwt/compare/2.3.0...2.4.0

2.3.0

What's Changed

... (truncated)

Changelog

Sourced from pyjwt's changelog.

v2.4.0 <https://github.com/jpadilla/pyjwt/compare/2.3.0...2.4.0>__

Security


- [CVE-2022-29217] Prevent key confusion through non-blocklisted public key formats. https://github.com/jpadilla/pyjwt/security/advisories/GHSA-ffqj-6fqr-9h24

Changed



- Explicit check the key for ECAlgorithm by @estin in https://github.com/jpadilla/pyjwt/pull/713
- Raise DeprecationWarning for jwt.decode(verify=...) by @akx in https://github.com/jpadilla/pyjwt/pull/742

Fixed
~~~~~

- Don't use implicit optionals by @rekyungmin in https://github.com/jpadilla/pyjwt/pull/705
- documentation fix: show correct scope for decode_complete() by @sseering in https://github.com/jpadilla/pyjwt/pull/661
- fix: Update copyright information by @kkirsche in https://github.com/jpadilla/pyjwt/pull/729
- Don't mutate options dictionary in .decode_complete() by @akx in https://github.com/jpadilla/pyjwt/pull/743

Added
~~~~~

- Add support for Python 3.10 by @hugovk in https://github.com/jpadilla/pyjwt/pull/699
- api_jwk: Add PyJWKSet.__getitem__ by @woodruffw in https://github.com/jpadilla/pyjwt/pull/725
- Update usage.rst by @guneybilen in https://github.com/jpadilla/pyjwt/pull/727
- Docs: mention performance reasons for reusing RSAPrivateKey when encoding by @dmahr1 in https://github.com/jpadilla/pyjwt/pull/734
- Fixed typo in usage.rst by @israelabraham in https://github.com/jpadilla/pyjwt/pull/738
- Add detached payload support for JWS encoding and decoding by @fviard in https://github.com/jpadilla/pyjwt/pull/723
- Replace various string interpolations with f-strings by @akx in https://github.com/jpadilla/pyjwt/pull/744
- Update CHANGELOG.rst by @hipertracker in https://github.com/jpadilla/pyjwt/pull/751

`v2.3.0 &lt;https://github.com/jpadilla/pyjwt/compare/2.2.0...2.3.0&gt;`__
-----------------------------------------------------------------------

Fixed
~~~~~

- Revert &quot;Remove arbitrary kwargs.&quot; `[#701](https://github.com/jpadilla/pyjwt/issues/701) &lt;https://github.com/jpadilla/pyjwt/pull/701&gt;`__

Added
~~~~~

- Add exception chaining `[#702](https://github.com/jpadilla/pyjwt/issues/702) &lt;https://github.com/jpadilla/pyjwt/pull/702&gt;`__

`v2.2.0 &lt;https://github.com/jpadilla/pyjwt/compare/2.1.0...2.2.0&gt;`__
-----------------------------------------------------------------------

&lt;/tr&gt;&lt;/table&gt; 
</code></pre>
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>

<ul>
<li><a href="https://github.com/jpadilla/pyjwt/commit/83ff831a4d11190e3a0bed781da43f8d84352653"><code>83ff831</code></a> chore: update changelog</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/4c1ce8fd9019dd312ff257b5141cdb6d897379d9"><code>4c1ce8f</code></a> chore: update changelog</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/96f3f0275745c5a455c019a0d3476a054980e8ea"><code>96f3f02</code></a> fix: failing advisory test</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/9c528670c455b8d948aff95ed50e22940d1ad3fc"><code>9c52867</code></a> Merge pull request from GHSA-ffqj-6fqr-9h24</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/24b29adfebcb4f057a3cef5aaf35653bc0c1c8cc"><code>24b29ad</code></a> Update CHANGELOG.rst (<a href="https://github-redirect.dependabot.com/jpadilla/pyjwt/issues/751">#751</a>)</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/31f5acb8fb3ec6cdfe2b1b0a4a8f329b5f3ca67f"><code>31f5acb</code></a> Replace various string interpolations with f-strings (<a href="https://github-redirect.dependabot.com/jpadilla/pyjwt/issues/744">#744</a>)</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/5581a31c21de70444c1162bcfa29f7e0fc86edda"><code>5581a31</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://github-redirect.dependabot.com/jpadilla/pyjwt/issues/748">#748</a>)</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/3d4d82248f1120c87f1f4e0e8793eaa1d54843a6"><code>3d4d822</code></a> Don't mutate options dictionary in .decode_complete() (<a href="https://github-redirect.dependabot.com/jpadilla/pyjwt/issues/743">#743</a>)</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/1f1fe15bb41846c602b3e106176b2c692b93a613"><code>1f1fe15</code></a> Add a deprecation warning when jwt.decode() is called with the legacy verify=...</li>
<li><a href="https://github.com/jpadilla/pyjwt/commit/35fa28e59d99b99c6a780d2a029a74d6bbba8b1e"><code>35fa28e</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://github-redirect.dependabot.com/jpadilla/pyjwt/issues/740">#740</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/jpadilla/pyjwt/compare/1.7.1...2.4.0">compare view</a></li>
</ul>
</details>

<br />




[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyjwt&package-manager=pip&previous-version=1.7.1&new-version=2.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---



Dependabot commands and options



You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/yassine-youcefi/web-scraping-nike-website/network/alerts).





Bump ipython from 7.16.1 to 7.16.3


opened on 2022-01-21 20:38:56 by dependabot[bot]

Bumps ipython from 7.16.1 to 7.16.3.



Commits







Dependabot compatibility score


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.





Dependabot commands and options



You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/yassine-youcefi/web-scraping-nike-website/network/alerts).





Hi this have this issue


opened on 2021-01-29 18:31:07 by etebanmp

uclient = ureq(my_url[x])



File "C:\python\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\python\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\python\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\python\lib\urllib\request.py", line 569, in error
    return self._call_chain(args)
  File "C:\python\lib\urllib\request.py", line 503, in _call_chain
    result = func(args)
  File "C:\python\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
yassine-youcefi

Youcefi Mohamme Yassine Full-Stack developer

GitHub Repository