JiKen is a simple kanji quiz that uses statistics and machine learning to accurately and quickly predict a user's knowledge level. I always found it tedious to get a good read of my current kanji level while studying using existing tests which either take forever or are terribly innaccurate so I made this.
The name JiKen is a bit of a play on words/kanji. It could be read as 字検 (letter test, similar to 漢検 the infamous official kanji test) or 事件 (incident). I left it in romaji for the ambiguity. KTest was just a working title, and kind of lame.
First thing to know to understand why this works so well is that kanji usage (and recognition) is not flat/random but has a relatively normal distribution and follows Ziph's Law. This allows us to make relatively sensible predictions of people's knowledge using a sigmoid function.
There are two main algorithms worth noting.
One predicts how many kanji you know (the graph) based on your answers. This is a Nelder-Mead regression algo with custom regularization: giving a lot of weight to the initial weights (safe assumption until data is collected), L2 reg (to avoid traps), some penalty to change between questions to give users a smooth experience. I also do bias correction as per https://cs.nyu.edu/~mohri/pub/bias.pdf since the questions selected are not random. No formal tuning methods were use, everything was done by hand until it felt good (the tuning target was to meet user expectations rather than simply being mathematically accurate).
The other algorithm ranks the difficulty of every kanji for future testing. If 100 people know "馬" but don't know "鹿" then the algorithm will shuffle the ranks around so that "鹿" is ranked lower, "馬" higher. This is called a Learning to rank algorithm: https://mlexplained.com/2019/05/27/learning-to-rank-explained-with-code/. Of course, this was again made more complicated by having biased sample selection.
You can report bugs here or contact me via reddit, twitter #jiken, or e-mail.
Shoot me a message if you want to do something with this code.
Bumps werkzeug from 0.15.5 to 2.2.3.
Sourced from werkzeug's releases.
2.2.3
This is a fix release for the 2.2.x release branch.
- Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-3
- Milestone: https://github.com/pallets/werkzeug/milestone/26?closed=1
This release contains security fixes for:
- https://github.com/pallets/werkzeug/security/advisories/GHSA-xg9f-g7g7-2323
- https://github.com/pallets/werkzeug/security/advisories/GHSA-px8h-6qxv-m22q
2.2.2
This is a fix release for the 2.2.0 feature release.
- Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-2
- Milestone: https://github.com/pallets/werkzeug/milestone/25?closed=1
2.2.1
This is a fix release for the 2.2.0 feature release.
- Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-1
- Milestone: https://github.com/pallets/werkzeug/milestone/24?closed=1
2.2.0
This is a feature release, which includes new features and removes previously deprecated features. The 2.2.x branch is now the supported bugfix branch, the 2.1.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as pip-tools to pin all dependencies and control upgrades.
- Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-0
- Milestone: https://github.com/pallets/werkzeug/milestone/20?closed=1
2.1.2
This is a fix release for the 2.1.0 feature release.
- Changes: https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-2
- Milestone: https://github.com/pallets/werkzeug/milestone/22?closed=1
2.1.1
This is a fix release for the 2.1.0 feature release.
- Changes: https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-1
- Milestone: https://github.com/pallets/werkzeug/milestone/19?closed=1
2.1.0
This is a feature release, which includes new features and removes previously deprecated features. The 2.1.x branch is now the supported bugfix branch, the 2.0.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as pip-tools to pin all dependencies and control upgrades.
- Changes: https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-0
- Milestone: https://github.com/pallets/werkzeug/milestone/16?closed=1
2.0.3
... (truncated)
Sourced from werkzeug's changelog.
Version 2.2.3
Released 2023-02-14
- Ensure that URL rules using path converters will redirect with strict slashes when the trailing slash is missing. :issue:
2533
- Type signature for
get_json
specifies that return type is not optional whensilent=False
. :issue:2508
parse_content_range_header
returnsNone
for a value likebytes */-1
where the length is invalid, instead of raising anAssertionError
. :issue:2531
- Address remaining
ResourceWarning
related to the socket used byrun_simple
. Removeprepare_socket
, which now happens when creating the server. :issue:2421
- Update pre-existing headers for
multipart/form-data
requests with the test client. :issue:2549
- Fix handling of header extended parameters such that they are no longer quoted. :issue:
2529
LimitedStream.read
works correctly when wrapping a stream that may not return the requested size in oneread
call. :issue:2558
- A cookie header that starts with
=
is treated as an empty key and discarded, rather than stripping the leading==
.- Specify a maximum number of multipart parts, default 1000, after which a
RequestEntityTooLarge
exception is raised on parsing. This mitigates a DoS attack where a larger number of form/file parts would result in disproportionate resource use.Version 2.2.2
Released 2022-08-08
- Fix router to restore the 2.1
strict_slashes == False
behaviour whereby leaf-requests match branch rules and vice versa. :pr:2489
- Fix router to identify invalid rules rather than hang parsing them, and to correctly parse
/
within converter arguments. :pr:2489
- Update subpackage imports in :mod:
werkzeug.routing
to use theimport as
syntax for explicitly re-exporting public attributes. :pr:2493
- Parsing of some invalid header characters is more robust. :pr:
2494
- When starting the development server, a warning not to use it in a production deployment is always shown. :issue:
2480
LocalProxy.__wrapped__
is always set to the wrapped object when the proxy is unbound, fixing an issue in doctest that would cause it to fail. :issue:2485
- Address one
ResourceWarning
related to the socket used byrun_simple
. :issue:2421
... (truncated)
22a254f
release version 2.2.3517cac5
Merge pull request from GHSA-xg9f-g7g7-2323babc8d9
rewrite docs about request data limits09449ee
clean up docsfe899d0
limit the maximum number of multipart form partscf275f4
Merge pull request from GHSA-px8h-6qxv-m22q8c2b4b8
don't strip leading = when parsing cookie7c7ce5c
[pre-commit.ci] pre-commit autoupdate (#2585)19ae03e
[pre-commit.ci] auto fixes from pre-commit.com hooksa83d3b8
[pre-commit.ci] pre-commit autoupdateDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase
.
Bumps certifi from 2019.6.16 to 2022.12.7.
9e9e840
2022.12.07b81bdb2
2022.09.24939a28f
2022.09.14aca828a
2022.06.15.2de0eae1
Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...b8eb5e9
2022.06.15.147fb7ab
Fix deprecation warning on Python 3.11 (#199)b0b48e0
fixes #198 -- update link in license9d514b4
2022.06.154151e88
Add py.typed to MANIFEST.in to package in sdist (#196)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase
.
Bumps numpy from 1.16.2 to 1.22.0.
Sourced from numpy's releases.
v1.22.0
NumPy 1.22.0 Release Notes
NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:
- Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
- A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
- NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
- New methods for
quantile
,percentile
, and related functions. The new methods provide a complete set of the methods commonly found in the literature.- A new configurable allocator for use by downstream projects.
These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.
The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.
Expired deprecations
Deprecated numeric style dtype strings have been removed
Using the strings
"Bytes0"
,"Datetime64"
,"Str0"
,"Uint32"
, and"Uint64"
as a dtype will now raise aTypeError
.(gh-19539)
Expired deprecations for
loads
,ndfromtxt
, andmafromtxt
in npyio
numpy.loads
was deprecated in v1.15, with the recommendation that users usepickle.loads
instead.ndfromtxt
andmafromtxt
were both deprecated in v1.17 - users should usenumpy.genfromtxt
instead with the appropriate value for theusemask
parameter.(gh-19615)
... (truncated)
4adc87d
Merge pull request #20685 from charris/prepare-for-1.22.0-releasefd66547
REL: Prepare for the NumPy 1.22.0 release.125304b
wipc283859
Merge pull request #20682 from charris/backport-204165399c03
Merge pull request #20681 from charris/backport-20954f9c45f8
Merge pull request #20680 from charris/backport-20663794b36f
Update armccompiler.pyd93b14e
Update test_public_api.py7662c07
Update init.py311ab52
Update armccompiler.pyDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase
.
Those <input type="text">
would be
This is important, as many Kanji have different shapes due to Han Unification. Chinese variant of a Kanji is of course outside the range a native, at least, should recognize the Kanji.