Releases: Privex/python-helpers
2.2.0 - Added export_private/public to KeyManager, add Mocker class plus more
The below list is mostly exhaustive list of all core changes in this version. Some small things may have been missed, but the majority
of changes are written below.
New Features / Additions
privex.helpers.crypto.KeyManager- Added
export_publicandexport_privateallowing you to export the currently loaded public/private key in KeyManager - Added
export_keywhich serialises a cryptography PrivateKey / PublicKey - intended for internal use by KeyManager, but
may be useful to others, so it's available as a public class method. - Refactored
generate_keypairto useexport_key - Added
identify_algorithmwhich returns the string algorithm e.g.'ed25519'for a given cryptography PublicKey / PrivateKey instance - Added attribute
type_name_mapwhich maps public/private key cryptography types to their string algorithm name - Improved exception handling in
load_keyensuring most common key decoding errors raiseInvalidFormatinstead of different package exceptions
- Added
- Added
Mockerclass tohelpers.common- useful for mocking classes, or used as a makeshift replacement for modules/classes that couldn't be imported - Added
mock_decoratortohelpers.decorators- intended to be used withMocker- it's a decorator which simply... does nothing. purely pass-thru - Added
HAS_CRYPTOHAS_SETUPPY_COMMONHAS_SETUPPY_BUMPandHAS_SETUPPY_COMMANDStohelpers.pluginallowing for easier detection whether certain
modules are available or not (e.g. due to a user not having a certain dependency package installed).
Testing
- Re-wrote parts of the "How to use the unit tests" docs in
tests/__init__.py - Added PyTest skipping to tests/test_cache.py for redis tests (uses the new
Mockerto ensure the tests work without PyTest installed) - Added PyTest skipping to tests/test_net.py for dnspython tests (uses
Mockerjust like test_cache) - Created a base class
CryptoBaseClassfor cryptography tests intest_crypto.py- Moved signing/verification into a helper method
_sign_verifyin this base class
- Moved signing/verification into a helper method
- Added new tests
- Split up KeyManager tests into
TestKeyManagerLoad,TestKeyManagerGeneration, andTestKeyManagerSignVerifyEncrypt - Added unit tests for outputting key pairs to files with
KeyManager.output_keypair - Added several unit tests for outputting and then loading key pairs from files with
KeyManager.load_keyfile - Added relatively thorough unit tests for the new
Mockerclass
- Split up KeyManager tests into
Documentation
- Lots of small changes.
- Removed
:noindex:from a lot of files because it was causing linking to classes/modules in pydoc comments to break.
This unfortunately means Sphinx throws a lot of warnings, but can't find a way to fix this without breaking class/module linking. - Added docs for
Mockerandmock_decorator - Re-generated toctree files for various modules including
cryptoandtests
2.1.0 Added human_name, refactored setuppy + more
This release is the official 2.1.0 release (as published to PyPi).
Melded commits from develop branch
Once we merged develop into master, two small fix commits were melded into this 2.1.0 commit, to ensure
the published PyPi 2.1.0 release matches this Git release.
-
fix all on dynamically loaded functions
-
forgot to include extras/tests.txt in dev.txt
New features:
-
Added new function
human_nametohelpers.commonwhich humanises a function/class name -
Converted
setuppy.pyinto a python module folder atsetuppy/ -
Added new distutils extras:
extras/setuppy.txtto cover all setuppy optional requirementsextras/docs.txtcontains requirements for building the Sphinx documentationextras/dev.txtis a meta-extra which includes all extra requirements for development viafull,docsandtests
-
Created
privex.helpers.setuppy.bump- Most notably, this module contains a function
bump_versionwhich uses the packagesemverto bump a version number.
The version number is extracted from a python package file configured insettings.VERSION_FILEand after it's updated,
the file containing the version is automatically updated with the new version number.
- Most notably, this module contains a function
-
Created
privex.helpers.setuppy.commandswhich contains setup.py / distutils command classes-
BumpCommandis a command class which allows you to bump the version of a python package, including updating the file containing the version,
simply by running a command such as./setup.py bump --minor -
ExtrasCommandis a command class which helps with managing your package'sextra_requireby offering these features:- With no arguments,
./setup.py extraswould output a list of all requirements listed for each extra in yourextras_require - With the
--save=somefile.txtargument, the requirements can be outputted into a file - With the
--installargument, the requirements will be installed withpipusing the python version which is running setup.py - With the
--extra=myextraargument, only requirements for that individual extra will be listed/saved/installed
- With no arguments,
-
-
Added unit tests for
human_nameintests/test_general.py -
Created
tests/test_net.pyfor network related unit tests
General changes / improvements:
-
Added
__all__to some modules, to reduce un-necessary objects being imported when you do wildcard imports such asfrom privex.helpers import * -
Moved
EXTRAS_FOLDERinto helpers.settings for consistency and easier changing -
Moved the various requirements.txt / extras related functions from
setuppy.pyintohelpers.setuppy.common -
Moved network related tests from
test_general.pyinto the newtest_net.py -
The
setup.pyused for privex-helpers itself now has the commandsbumpandextrasavailable, which uses the
new command classes fromprivex.helpers.setuppy.commands -
The file
privex/helpers/__init__.pynow importsprivex.loghelperinside of a try/except block, so if for some strange reason
privex-loghelperisn't installed or is broken, then it will fallback to standardlogging.getLoggerinstead of just breaking -
Fixed various issues with PyDoc blocks in several files, such as
cache.KeyManagerandcrypto.RedisCache -
Travis-CI now uses
pip install -U '.[dev]'instead of installing the requirements filedocs/requirements.txtand package -
ReadTheDocs now uses
pip install -U '.[dev]'instead of installing the requirements filedocs/requirements.txt -
Possibly other small fixes / changes
Documentation related:
-
Listed some of the new
setuppyfunctionality in the overview in README.md -
Updated the
docs/requirements.txtfile to include all important requirements -
Updated the
module.rstautosummary template to include attributes (if possible) -
Updated the
class.rstautosummary template so inherited attributes / methods are ignored (generated unit test class docs were often really spammy) -
Added detailed documentation for the
privex.helpers.setuppymodule -
Added an autosummary block for
privex.helpers.settingsso that it can correctly show the settings in the navigation -
Fixed up several module/class rst files with
:noindex:and other minor fixes to reduce the amount of warnings when building the docs -
Re-organised the navigation tree for some modules to make it easier to use
v2.0.0 MAJOR - Added KeyManager class, refactored crypto module, documentation changes, packaging changes and more!
There have been many, many changes since version 1.0, so I've decided it's about time that we finally bump our major version to version 2.0.0
While this release isn't really a breaking change from 1.5.0, there have been so many breaking changes introduced in the 1.x.x series, as well as major new features such as a cache abstraction layer and a cryptography module, it's worth having a clean semver slate with 2.0.0
Docs
- Added documentation for
byteifyandstringify - Added documentation for the
cryptomodule - Added documentation for the
setuppymodule - Updated the custom sphinx CSS to allow for a sensible 5th nesting level in the navigation
- Adjusted the sphinx templates for
autosummary/class.rstandautosummary/module.rst- Insert toctree directive automatically into generated files, so sub-components automatically have their RST stubs for navigation generated,
instead of having to manually modify the generate files afterwards.:toctree:is automatically added for module functions+classes:toctree:is automatically added for class methods+attributes
- The headings "Methods" and "Attributes" are automatically added for classes, for display in the navigation bar.
- Insert toctree directive automatically into generated files, so sub-components automatically have their RST stubs for navigation generated,
- Added
sphinx_rtd_themeas an import and in the extension list inconf.py(the README says to do this, so maybe it will prevent issues in the future?)
Packaging
- Added
extras_requireto setup.py, allowing "extras" to be specified when installing privex-helpers, instead of
having to specify optional dependencies directly in your requirements.txt - Excluded the
testsmodule from the package
Unit Tests
- Created
TestKeyManagerclass inside oftests/test_crypto.pywhich tests the new asymmetric key system- Tests generation of RSA, ECDSA and Ed25519 keys by checking the public/private key length, as well as looking for strings in their output format
- Tests loading of RSA, ECDSA, and Ed25519 keys
- Tests loading invalid keys correctly raises an exception (otherwise how do we know that the previous loading attempts weren't just being ignored?)
- Tests signing and verifying messages for RSA, ECDSA and Ed25519 keys
- Tests encryption and decryption for RSA keys (other algorithms don't support encryption/decryption)
Code
cryptomodule is now a folder python module- Moved EncryptHelper into crypto/EncryptHelper.py
- Created crypto/base.py with helper functions such as
is_base64andauto_b64decode - Created crypto/KeyManager.py
- Generates Asymmetric keys in RSA, ECDSA and Ed25519 format
- Serialize public/private keys into various formats, default: PEM+PKCS8 for private, OpenSSH for public
- Load public/private keys in most formats, with auto detection
- Ability to load a key from a file
- Automatically interpolate public key when loading a private key
- Ability to output generated keys to private and public file
- Signing messages with RSA, ECDSA, and Ed25519
- Verifying messages with RSA, ECDSA, and Ed25519
- Encrypting messages/data with RSA
- Decrypting messages/data with RSA
- Lots of PyDoc blocks, clearly documenting how everything works
- New
setuppymodule with various packaging related helpers, such as parsing requirements files and handling importing other requirements files - Added
InvalidFormatexception, used by the crypto module commonmodule- New
byteifyfunction - convert a value into bytes if it isn't already bytes - New
stringifyfunction - decode bytes into a string, if it isn't already a string
- New
v1.5.0 - Added `crypto` module + docs improvements
- Added
cryptomodule with newEncryptHelperclass, designed to make using encryption
viacryptography.fernetpainless as possible. - Added docs for
cryptomodule - Set up InterSphinx, allowing for cross referencing several different package docs
- Added new exceptions
EncryptionErrorandEncryptKeyMissing- related tocryptomodule - Added several unit tests for the
cryptomodule intest_crypto.py
v1.4.0 (BREAKING CHANGES) - Added `inject_items`, tests overhaul, bug fixes and cleanup
Notable Changes
New Features
- Added
common.inject_items- a small function for injecting a list into another list
at an arbitrary position.
General Improvements/Fixes
- Fixed
net.asn_to_nameso that it correctly raisesKeyErroror returns
'Unknown ASN'when dnspython raisesNoAnswer - Added
plugin.HAS_DNSPYTHONattribute, for checking whether the dnspython dependent
functions/classes were loaded or not - (POTENTIALLY BREAKING CHANGE)
__init__.pyno longer loads thedjangomodule,
as it was causing endless issues related to Django not being configured, or
not being ready for accessing certain modules.
The change to __init__.py loading Django means importing privex.helpers.django functions/classes via from privex.helpers import xxx will no longer work.
Any application which imported django functions/classes via privex.helpers must now use absolute imports, e.g.
from privex.helpers.django import is_database_synchronized
Documentation
- Fixed small typo in
cache.RedisCachepydoc block - Re-organised Unit Tests section to make it more readable
- Re-named the cache module into "Cache Abstraction Layer", as the module page contains
a lot of hand written documentation about it. - Re-organised the code documentation to be top level
- Added Usage Examples section
- Fleshed out the index page of the docs
Unit Test updates
- Extracted tests from
tests.pyinto individual files intestsfolder, as tests.py
was close to 500 lines long and growing... - Added unit tests for
net.asn_to_name - Added unit tests for
common.inject_items
Added new `cache` module to privex.helpers, overhauled docs
TL;DR; Created new module privex.helpers.cache for easy framework-agnostic caching, plus cleaned up many page titles in the docs
Caching:
- The main
privex.helpers.cachemodule exposes acachedattribute (from privex.helpers import cached) which
acts as a singleton adapter wrapper, so the global cache adapter can easily be switched out without any risk
of other parts of your code using the "old" adapter. - Created
CacheAdapterwhich is an abstract base class designed to set the standard for the cache API - Created
MemoryCachewhich is a simple caching layer with expiration support that simply stores cache items in memory
using a staticdictattribute. - Created
RedisCachewhich as it sounds, is a cache layer that uses Redis for it's backend. It uses the global Redis fromprivex.helpers.pluginby default,
however you can pass aredis.Redisclass instance to it's constructor to use a custom instance instead.
Unit testing:
- Created unit tests for
MemoryCache - Created unit tests for
RedisCache- based on the unit tests forMemoryCacheto avoid code duplication
Other Updates:
- Added two new functions to
privex.helpers.pluginconfigure_redisfor updating the global redis settings and automatically replacing the Redis instancereset_redisto close the current Redis connection, then delete and re-instantiate the Redis connector class.
- Added two new exceptions
CacheNotFoundandNotConfigured - Possibly some other small changes
Documentation
- Added very detailed documentation for the new
privex.helpers.cachemodule - Added documentation for
CacheAdapter - Added documentation for
MemoryCache - Added documentation for
RedisCache - Fixed some missing documentation entries
- Added slightly customized autosummary templates under
docs/source/_templatesto stop Sphinx using
ridiculously long namespaced methods/attributes for the page titles / navbar. - Regenerated lots of stub files in
docs/source/helpers/such ascommon/,exceptions/,net/, and others - fixes the long title issue permanently (see previous bullet point)
v1.3.0 - Added `async_sync`, `run_sync` + More
asyncx.py
- Added
async_syncdecorator for easily running async functions/methods from
non-async functions/methods. Useful for unit tests. - Added
run_syncfunction, mainly intended for running async functions from the REPL with ease
common.py
- Added
dec_roundfor roundingDecimalobjects to arbitrary DP - Added
env_cast,env_int, andenv_decimal - Added
chunkedfunction for slicing a list into multiple equal sized lists - Added
Dictableabstract class - eases creating Python 3.7dataclassobjects fromdict's
as well as converting them back intodict's
tests.py
- Added new
TestGeneraltest case, for unit testing some of the new functions
Documentation
- Added docs for
asyncxmodule - Updated
commonmodule to include new functions/classes - Fixed docs for
tests - Various cleanup to reduce warnings from Sphinx
Added `env_bool` function, csv/kval functions now allow separator to be overriden, + more
New functions / code improvements
- Added
env_boolfunction tocommon.pyfor loading an environment var as a boolean - Added
csvsplitparameter to parse/env_csv and keyval, to allow changing the item separator from, - Added
valsplitparameter to parse/env_keyval, to allow customising the separator between key's and value's from:
Documentation
- Added PyDoc param's / return's for various functions, and fleshed out some others
- Wrapped various docstring values such as
Trueand0with backticks so they display better - Various small formatting improvements to existing docstrings
- Added
PrivexBaseCaseandenv_boolto the docs
Unit Testing
- Refactored various attributes in
test.pyinto the base classPrivexBaseClass - Added example to the PyDoc in
tests.pyshowing how to run tests withpytest - Wrote new unit tests:
test_kval_custom_clean- Validates thatparse_keyvalworks properly with customvalsplit/csvsplittest_kval_custom_spaced- Validates thatparse_keyvalworks properly with space padded values and customvalsplit/csvsplittest_env_nonexist_bool,test_env_bool_true,test_env_bool_false- Validate that the newenv_boolfunction returns the correct values.
Added rDNS functions, exceptions, and more.
- Added various rDNS functions to
privex.helpers.netsuch asip_to_rdns - Added unit tests to
tests.pyfor the rDNS functions - Created new
exceptions.pywith various exception classes - Fixed
setup.pyto use BASE_DIR, avoiding issues if setup.py is ran relatively - Fixed django.py imports
- Cleaned up PyDoc comments for modules and some functions
- Added PyDoc comment to tests.py explaining how to use the unit tests
- Various other small improvements
Initial release
Added the initial versions of the following helper functions/classes:
- Common helpers:
random_str,empty,is_true,is_false,ErrHelpParser - Decorators:
retry_on_err - Django helpers:
handle_error,is_database_synchronized,model_to_dict,to_json - Net helpers:
asn_to_name,ip_is_v4,ip_is_v6
Also added basic usage information in README.md