twine (the tool we use to upload packages to PyPI) is currently failing if the long_description (used to render a project's page on PyPI website) is not set. Somehow it complains that it is not well formatted reStructuredText, but it's simply empty...
This looks like a bug, or bad interaction between twince and setuptools, because the field is technically optional.
Also see https://github.com/pypa/twine/issues/960 and https://github.com/pypa/twine/issues/908.
This issue is currently preventing the upload of newly built Brotli v1.1.0 Python wheels to PyPI:
https://github.com/google/brotli-wheels/issues/18#issuecomment-1706910190
Anyway, we may well set the long_description to the content of the README.md file, as it's customary for python projects.
Currently, the output buffer is a std::vector<uint8_t>.
When the buffer grows, resizing will cause unnecessary memcpy().
This change uses a list of bytes object to represent output buffer, can avoid the extra overhead of resizing.
In addition, C++ code can be removed, it's a pure C extension.
* IMPORTANT: decoder: fix potential overflow when input chunk is >2GiB
* simplify max Huffman table size calculation
* eliminate symbol duplicates (static arrays in .h files)
* minor combing in research/ code
Our dist tarball is missing hash_rolling_inc.h and
hash_composite_inc.h, which causes subsequent autotools
builds to fail. Fix this by adding it to the sources list.
Signed-off-by: William A. Kennington III <william@wkennington.com>
* New feature: "Large Window Brotli"
By setting special encoder/decoder flag it is now possible to extend
LZ-window up to 30 bits; though produced stream will not be RFC7932
compliant.
Added new dictionary generator - "DSH". It combines speed of "Sieve"
and quality of "DM". Plus utilities to prepare train corpora
(remove unique strings).
Improved compression ratio: now two sub-blocks could be stitched:
the last copy command could be extended to span the next sub-block.
Fixed compression ineffectiveness caused by floating numbers rounding and
wrong cost heuristic.
Other C changes:
- combined / moved `context.h` to `common`
- moved transforms to `common`
- unified some aspects of code formatting
- added an abstraction for encoder (static) dictionary
- moved default allocator/deallocator functions to `common`
brotli CLI:
- window size is auto-adjusted if not specified explicitly
Java:
- added "eager" decoding both to JNI wrapper and pure decoder
- huge speed-up of `DictionaryData` initialization
* Add dictionaryless compressed dictionary
* Fix `sources.lst`
* Fix `sources.lst` and add a note that `libtool` is also required.
* Update setup.py
* Fix `EagerStreamTest`
* Fix BUILD file
* Add missing `libdivsufsort` dependency
* Fix "unused parameter" warning.
* [appveyor] remove 'deploy' stage; only test python 2.7 and 3.6
all the other python versions are being built and tested on
https://github.com/google/brotli-wheels/blob/d571d63/appveyor.yml
* remove terrify submodule as not needed any more
* [travis] just test py2.7 and 3.6 on linux; remove extra osx python builds
All the other python versions for OSX are being built/tested on:
https://github.com/google/brotli-wheels/blob/d571d63/.travis.yml
Also, there's no need to build and deploy wheels here, as that's done
in the separate repository.
* [setup.py] only rebuild if dependency are newer; fix typo in list of 'depends'
https://github.com/python/cpython/blob/v3.6.2/Lib/distutils/command/build_ext.py#L485-L500
* [ci] only run 'python setup.py test'
if we run 'python setup.py built test', the setuptools 'test' command will
forcibly re-run the build_ext subcommand because it wants to pass the --inplace
option (it ignores whether it's up to date, just re-runs it all the time).
with this we go from running built_ext twice, to running it only once per build
* [Makefile] run 'build_ext --inplace' instead of 'develop' as default target
The 'develop' command is like 'install' in the sense that it
modifies the user's python environment.
The default make target should be less intrusive, i.e. just building
the extension module in-place without modify anything in the user's
environment.
We don't need to tell make about the dependency between 'test' and
'build' target as that is baked in the `python setup.py test` command.
* [Makefile] add 'develop' target; remove unnecessary 'tests' target
`make test` is good enough
* [Makefile] `setup.py test` requires setuptools; run `python -m unittest`
This will work even if setuptools is not installed, which is unlikely
nowadays but still our `setup.py` works with plain distutils, so
we may well have our tests work without setuptools.
* [python/README.md] add ref to 'develop' target; remove 'tests', just 'make test'
* [setup.py] import modules as per nicksay's comment
https://github.com/google/brotli/pull/583#discussion_r131981049
* [Makefile] add 'develop' to .PHONY targets
remove 'tests' from .PHONY
* [appveyor] remove unused setup scripts
We don't need to install custom python versions, we are
using the pre-installed ones on Appveyor.
* [appveyor] remove unneeded setup code
Common:
* wrap dictionary data into `BrotliDictionary` structure
* replace public constant with getter `BrotliGetDictionary`
* reformat dictionary data
Decoder:
* adopt common changes
* clarify acceptable instance usage patterns
* hold reference to dictionary in state
Encoder:
* adopt common changes
* eliminate PIC spots in `CreateBackwardReferences`
* add per-chunk ratio guards for q0 and q1
* precompute relative distances to avoid repeated calculations
* prostpone hasher allocation/initialization
* refactor Hashers to be class-like structure
* further improvements for 1MiB+ inputs
* added new hasher type; made hashers more configurable
Java:
* Pull byte->int magic to `IntReader` from `BitReader`
* pull `BROTLI_MAX_BACKWARD_LIMIT` to constants
* split generic and Zopfli backward references code
* pull hashers init and stitch invocation to encoder
* make `dictionary_hash` a compilation unit
* add `size hint` parameter
* add new hasher
* use `size hint` to pick new hasher for q4
* modernize clz guard (fix#495)
* move `hash to binary tree` to separate file
* add `Initialize` and `Cleanup` to all hashers
* do not raise OOM if malloc(0) == NULL (fix#500)
