Releases: deepjavalibrary/djl
Releases · deepjavalibrary/djl
DJL v0.19.0 release
Key Features
- Creates new LightGBM engine (#1895)
- Upgrades and enhancements for Engines
- Introduces several enhancement for HuggingFace tokenizer
- Introduces HuggingFace model zoo (#1984)
- Adds a few built-in Translators for HuggingFace NLP models
- Adds macOS M1 support for HuggingFace tokenizer
- Adds ARM support for HuggingFace tokenizer
- Adds centos 7 support for HuggingFace tokenizer (#1874)
- Adds decode API for HuggingFace tokenizer (#1843)
- Adds padding and truncation support for HuggingFace tokenizer (#1870)
- Support stride in tokenizers (#2006)
- Introduces time series extension (#1903)
- Adds new Audio API and improves audio extension (#1974)
- Adds Android support for ONNXRuntime (#1844)
- JDK18 support (#1892)
- Adds python script to import HuggingFace model into DJL model zoo (#1835)
- DJLServing
Enhancement
- Adds a few more built-in Translators:
- Adds new models to DJL model zoo:
- Image handling enhancement:
- DJL API improvements:
- Adds NDArray normalize() operator (#1924)
- Adds DeferredTranslatorFactory to let serving.properties take effect (#1868)
- Makes PtBertQATranslator compatible with huggingface model (#1827)
- Improves debug log for model loading options. (#1825)
- Allows to load block only model for PyTorch (#1831)
- Adds IdentityBlockFactory for demo/test purpose (#1854)
- Support queryString for JarRepository (#1842)
- Set arguments in serving.properties as model properties (#1853)
- Allow overriding special token flags in encode and decode methods (#1855)
- Adds support for intermediate sequential block results (#1943)
- Adds load SentencePiece model from InputStream (#1949)
- Allows use cached PyTorch native libraries in offline mode by caching "files.txt". (#1982)
- Makes Encoding class constructor protected. (#1945)
- Adds string tensor support for PyTorch (#1968)
- Adds Loss function: Coverage.java (#1653)
- Adds Loss function: QuantileLoss.java (#1652)
- Validate data type for NDArray.set(Buffer) API (#1975)
- Adds offline mode to to ensure not download engine files from network (#1987)
- Adds encodeDual support for HuggingFace tokenizer (#1826)
- Bulk batch creation and array indexing on mxnet engine (#1869)
- Adds NDArray gammaln and sample distribution function support. (#1990)
- Padding when the size of input is 2 in LSTM (#2000)
- Creates a SystemNDManager interface (#1888)
- Adds python script to import huggingface model into DJL model zoo
- Added fill-mask support for converting huggingface model to model zoo (#1849)
- Adds support for converting huggingface token-classification models (#1902)
- Adds support for converting huggingface sentence-similarity models (#1913)
- Adds support for converting huggingface text-classification models (#1972)
Documentation and Examples
- Adds Neural machine translation example (#1851)
- Adds New Bert example using Goemotions (#1682)
- Adds Semantic Segmentation example (#1808)
- Adds tokenizer readme for usage (#1981)
- Updates troubleshooting.md to remove -native-auto package (#1793)
- Document PYTORCH_PRECXX11 usage in README (#1807)
- Immutable array output from InferenceMode PyTorch (#1822)
- Fixes NDIndex javadoc issue (#1841)
- Updates pose estimation example to detect joints for all people (#2002)
- Adds Semantic segmentation and Speech recognition to README (#2003)
- Updates links in README (#2005)
- Adds an example of time series model inference (#1971)
Breaking change
- NDManager.vaildateBufferSize() has been renamed to NDManager.validateBuffer()
- Remove unnecessary DeviceType interface (#1978)
Bug Fixes
- Adds missing text_embedding application in Application.of() (#1917)
- Fixes capped manager bug for TensorFlow (#1952)
- Fixes NDArray.set() bug (#1789)
- Fixes breaking behavior for NDIndex in 0.18.0 (#1801)
- Backward compatible with Apache MXNet indexing. (#1802)
- Fixes OrtNDArray double close issue (#1809)
- Fixes ImageFactory.fromNDArray() bug (#1824)
- Fixes NDArrayAdapter toDevice() and toType() behavior (#1839)
- Fixes the parsing issue (#1857)
- Fixes OrtNDArray double free issue (#1861)
- Fixes memory leak when using NDManager.newBaseManager() (#1887)
- Fixes PyTorch download library fallback to CPU bug (#1951)
- Fixes bug in Criteria (#1964)
- Fixes issue in TrainMnistWithLSTM (#1965)
- Fixes...
Contributors
zachgk, pdradx, and 17 other contributors
Assets 2
DJL v0.18.0 release
Key Features
- Adds macOS M1 chip support for PyTorch #1656, #1696
- JDK 17 support #1672
- Full support of PyTorch Get Indexing for NDArrays #1719
- Full support of PyTorch Set Indexing for NDArrays #1755
- Updates Dataset documentation #1686
- Moves djl-bench to DJL Serving #1743
- Engines and Extensions
- Newly Added Datasets
Enhancement
- Increases build version to 0.18.0 #1645
- Support of take from pytorch #1627
- Upgrades JNA to 5.11.0 #1655
- Improves ServingTranslator output handling #1654
- Adds width/height conversion to ObjectDetection #1651
- Add openCV find rectangle method to improve PaddleORC performance #1662
- Removes unnecessary logics in Paddle #1676
- Adds Cyclical Tracker #1671
- Adds support of
takeon MXNet engine #1649 - Implements GhostBatchNorm #1666
- Allows indexer to attach specific manager #1688
- Upgrades android module to use DJL 0.18.0 #1693
- Uses pytorch to test API and aws-ai module #1695
- Avoid download cudf dependency for XGBoost at build time #1694
- Bumps up versions #1691
- Refactors ServingTranslatorFactory #1702
- Adds "capped" state to NDManager #1683
- Upgrades NDK version to 21.1.6352462 #1707
- Adds LinearCollection block #1658
- Adds android test code #1714
- Changes DJL repo names from aws-samples #1716
- Adds serving deb file publish for CI #1721
- Upgrades codeql github action to v2 #1730
- Fixes publish serving deb #1725
- Adds ai.djl.audio and ai.djl.tablesaw to BOM #1728
- Upgrades java formatter to 1.15.0 #1727
- Adds name to LambdaBlock #1726
- Adds disable static option in MXNet to allow some model running #1735
- Improves Criteria.toBuilder() api #1741
- Fixes serving publish github actions #1742
- Enables better textual description of neural net #1720
- Ignores hidden files for nested model directory #1754
- Creates action to auto-close issues without response #1751
- Builds jni for aarch64 #1756
- Removes unnecessary packages from tensorrt dockerfile #1760
- Adds log for custom Translator loading #1761
- Stores indices with batch #1750
- Adds
putfeature with linear indexing on PyTorch engine #1749 - Adds NDList to IValue unit test #1762
- Makes tensorflow NDArray always dense #1763
- JDK version updated #1767
- Adds IValue Dict(str, IValue) support #1765
- Creates tabular dataset #1699
- Creates PreparedFeaturizer #1700
- Normalizes Numeric Featurizer #1701
- Adds support for registerCustomOpLibrary for ONNXRuntime. #1771
- Implements inverse operation #1768
- Supports Image output for ImageServingTranslator #1772
- Allows user specify model name in serving.properties file #1780
- Adds model zoo implementation #1781
- Change the sagemaker model to s3 #1769
- Improvements to image coloring #1784
- Updates bert classification notebook to reflect changes in CSVDataset #1786
- Paddle model zoo should not have compile time dependency on opencv #1785
Documentation and Examples
- Updates README for 0.17.0 Release #1646
- Increases DJL Version for main branchhttps://github.com//pull/1644
- Fixes broken and redirected links #1647
- Clarifies typo in example documentation #1685
- Fixes javadoc error in JDK 1.8 #1698
- Update description for latest javadoc location #1708
- Creates README for DJL Android PyTorch 1.11 builds #1704
- Adds serving to docs site #1715
- Fixes broken javadoc links in jupyter notebooks #1722
- Readme updates for PyTorch 1.11 #1709
- Updates CVSDataset example README file #1729
- Updates document to use MXNet 1.9.0 #1737
- Adds documentation on loading TF extension libraries for running certa… #1776
- Adds semantic segmentation example #1764
Breaking Changes
The following changes to api.djl.basicdataset.tabular may cause backwards incompatibility:
- Features and Featurizers have been refactored out of the CSVDataset class. The are now present in ai.djl.basicdataset.tabular.utils
- CSVDataset now extends a new abstract class, TabularDataset
- api.djl.basicdataset.utils.DynamicBuffer implementation has moved to api.djl.basicdataset.tabular.utils.DynamicBuffer
Bug Fixes
- [TensorFlow] fix GPU memory leak #1648
- [tensorrt] Fixes native library path #1650
- Fixes bug in NDArray.oneHot() API #1661
- Fix errors in "getIoU" function #1687
- Follow symlinks when loading models. #1692
- [pytorch] Fixes model loading bug for 1.11.0 #1705
- Ensure PreparedOneHotStringFeaturizer encodes categorical mappings co… #1723
- [tensorflow] Avoid NPE in TfEngine #1734
- [m1] Fix test failure on macOS M1 machine by @frankfliu in #1777
Contributors
@dandansamax
@DiaaAj
@frankfliu
@WHALEEYE
@patins1
@JohnDoll2023
@KexinFeng
@Konata-CG
@pdradx
@lanking520
@siddvenk
@LanAtGitHub
@warthecatalyst
@zachgk
@freemanliu
@liumingxiy
New Contributors
- @Konata-CG made their first contribution in #1598
- @pdradx made their first contribution in #1661
- @JohnDoll2023 made their first contribution in #1685
- @liumingxiy made their first contribution in #1687
- @LanAtGitHub made their first contribution in #1679
- @freemanliu made their first contribution in #1692
- @DiaaAj made their first contribution in #1666
- @warthecatalyst made thei...
Contributors
zachgk, pdradx, and 14 other contributors
Assets 2
DJL v0.17.0 release
Key Features
- Adds linux AArch64 support for PyTorch
- Upgrades Engine Releases
- Newly Added Datasets
- Support freezing parameters for transfer learning (#1544)
Enhancement
- Upgrade djl-bench to 0.15.0 (#1476)
- Add SessionOptions support for OnnxRuntime (#1479)
- Device Name parsing (#1490)
- arena allocator setter (#1510)
- Upgrade native build mxnet version (#1517)
- Refactor ml extension support (#1521)
- Parse default device with Device.fromName (#1529)
- Upgrade dependency versions (#1533)
- release Update document version to 0.16.0 (#1536)
- Bump up version to 0.17.0 (#1537)
- Building DJL For aarch64 (#1526)
- api Make resource loading compatible with java9 module (#1541)
- pytorch Allows multiple native jars to package into fat jar (#1543)
- Add document with installation instructions (#1548)
- Separate AbstractSymbolBlock from AbstractBlock (#1555)
- Add better error message for libstdc++ tf errors (#1570)
- basicdataset Add Stanford Question Answering Dataset (#1554)
- xgb Set default missing value to NaN (#1571)
- allow empty batch data (#1569)
- pytorch Allows load libtroch from pip installation package (#1577)
- HF Tokenizer: get charspans (#1584)
- benchmark Allows benchmark run on aarch64 for PyTorch (#1591)
- add pytorch cuDNN acceleration (#1592)
- add troubleshoot issue (#1600)
- add paddle CU110 (#1604)
- Update badge (#1610)
- ONNXRuntime add tensorRT option (#1611)
- api Refactor metrics API (#1613)
- api Fixes metric dimension (#1614)
- bom Update bom dependency (#1615)
- pytorch Use precxx11 build for aarch64 native library (#1619)
- pytorch use precxx11 for aarch64 (#1620)
- Update inference_performance_optimization.md (#1621)
- Fix testFreezeParameters for multi-gpu (#1623)
- Support gather of pytorch (#1622)
Documentation and Examples
- Update README for 0.15.0 release (#1477)
- Update README to use 0.16.0-SNAPSHOT version (#1486)
- typo fix (#1519)
- docs Fixes dataset document. (#1523)
- jupyter remove unecessary maven import (#1540)
- examples Update maven pom file dependency version (#1546)
- Add release note to README (#1565)
- update pytorch build instruction on android (#1630)
- bump up versioning (#1633)
- update android package versions (#1635)
- docs Update pytorch and paddle version (#1634)
Breaking Changes
- Custom symbol blocks should extend
AbstractSymbolBlockinstead ofAbstractBlock
Bug Fixes
- Fix flaky test for tensorflow 2.7.0 on GPU (#1475)
- Fixes topK items for DetectedObjects and make it configurable to Classifications (#1478)
- Fix load native library failure on Android (#1485)
- Adding huggingface tokenizer extension to BOM (#1487)
- Fixes #1149, fix NPE bug (#1492)
- Fix djl site vue version (#1495)
- Fixes MLP example code in README (#1497)
- Fixes jni dependency in README document (#1513)
- Fixes memory leak in hybrid engine (#1518)
- Fix the version issue reported from get-pip.py (#1530)
- Fix FastText JNI build (#1531)
- api Fixes loading BlockFactory bug (#1547)
- tensorflow Fixes tensorflow session always on gpu(0) bug (#1558)
- xgb Fixes missing anonymous classes (#1572)
- examples Fix ImageClassification invalid probability (#1575)
- fix naming (#1581)
- ONNXRuntime fix naming (#1608)
- basicdataset Fixed out of bound limit (#1599)
- api Avoid NPE in Metric.toString() (#1626)
- integration Enable gather unit test for windows (#1638)
- Add rpath fix to Native Publish PyTorch (#1639)
- api Fixes JDK 18 compiler warnings (#1640)
Contributors
- @AKAGIwyf
- @andreabrduque
- @dandansamax
- @frankfliu
- @hd1080p
- @KexinFeng
- @lanking520
- @patins1
- @sindhuvahinis
- @WHALEEYE
- @zachgk
New Contributors
- @sindhuvahinis made their first contribution in #1519
- @KexinFeng made their first contribution in #1530
- @hd1080p made their first contribution in #1526
- @dandansamax made their first contribution in #1545
- @WHALEEYE made their first contribution in #1554
- @patins1 made their first contribution in #1569
- @AKAGIwyf made their first contribution in #1580
Full Changelog: 750c153...v0.17.0
Contributors
zachgk, patins1, and 9 other contributors
Assets 2
DJL v0.16.0 release
Key Features
- Upgrades Apache MXNet engine to 1.9.0 with CUDA 11.2 support
- Improves ONNXRuntime engine memory configurations
- Improves fastText engine’s API
- Fixes several critical bugs
Enhancement
- Upgrades Apache MXNet engine to 1.9.0 with CUDA 11.2 support (#1517)
- Improves ONNXRuntime engine:
- Introduces several API improvements:
- Uses JDK 11 for github actions in CI build (#1489)
- Adds huggingface tokenizer extension to BOM (#1487)
- Removes unnecessary github actions workflow (#1484)
- Upgrades DJL android to 0.15.0 (#1483)
Documentation and examples
- Fixes outdated dataset document (#1523)
- Fixes repository README typo (#1519)
- Fixes PyTorch JNI dependency in README document (#1513)
- Fixes MLP example code in README (#1497)
- Fixes djl.ai website vue version (#1495)
- Updates inferentia demo to DJL 0.15.0 (#210)
- Updates android README to use 0.15.0 (#1486)
- Publishes D2L Chinese book with latest chapters https://d2l-zh.djl.ai
Breaking change
- fastText specific inference and training APIs are removed, use standard DJL API instead
Bug Fixes
- Fixes memory leak in hybrid engine (#1518)
- Fixes fastText JNI build (#1531)
- Fixes the python version bug in the benchmark workflow files (#1530)
- Fixes sentencepiece NPE bug (#1492)
- Fixes PyTorch load native library failure on Android (#1485)
- Fixes topK items for DetectedObjects (#1478)
Contributors
This release is thanks to the following contributors:
- Andréa Duque (@andreabrduque)
- Dennis Kieselhorst (@deki)
- Frank Liu (@frankfliu)
- Jake Lee (@stu1130)
- Kexin Feng (@KexinFeng)
- Qing Lan (@lanking520)
- Sindhu Somasundaram (@sindhuvahinis)
- Zach Kimberg (@zachgk)
New Contributors
- Andréa Duque made their first contribution in #1510
Full Changelog: https://github.com/deepjavalibrary/djl/compare/v0.15.0..v0.16.0
DJL v0.15.0 release
DJL v0.15.0 updates the engines PyTorch to 1.10.0, ONNXRuntime to 1.10.0, TensorFlow to 2.7.0, TensorFlowLite to 2.6.2 and introduces several new features:
Key Features
- Introduces Huggingface tokenizers extension which allows user to leverage high performance fast tokenizer in Java
- Upgrades PyTorch engine to 1.10.0 with CUDA 11.3 support
- Upgrades TensorFlow to 2.7.0 with CUDA 11.3 support
- Upgrades ONNXRuntime engine to 1.10.0
- Upgrades TensorFlowLite to 2.6.2
- Provides better PyTorch engine backward compatibility support
- Adds load model from InputStream support
- Adds Windows support for SentencePiece
- Removes -auto packages to simplify DJL dependencies
- Fixes log4j CVEs
Enhancement
- Improves PyTorch engine:
- Adds support to load custom build PyTorch native library from specified location
- Adds support to use precxx11 version of PyTorch on GPU
- Provides offline native package for older version of PyTorch to run with latest DJL (#1385)
- Report correct engine version for custom build PyTorch native library
- Adds Tuple support to IValue (#1436)
- Adds support to load model from InputStream for some of the engines (#1400):
- Introduces several new features in djl-serving:
- Automatically detect model’s engine
- Reduces netty dependencies to minimize package size
- Installs engine dependency on-demand for non-commonly used engines
- Adds support for nested folder in model archive file
- Released djl-serving to homebrew
- Introduces server new features in djl-bench
- Release djl-bench to homebrew and snapcraft
- Improve opencv extension to expose API from opencv to end users
- Introduces several API improvements:
- Publishes several new docker images to dockerhub (https://hub.docker.com/u/deepjavalibrary)
- djl-serving docker image
- djl-serving for inferentia docker image
- DJL windows docker image with jdk11
Documentation and examples
- Updates inferentia demo to support neuron runtime 2.x
- Updates jupyter notebooks with NoBatchifyTranslator to simplify example code (#1370)
- Updates benchmark README (#1410)
- Fixes rank classification jupyter notebook (#1368)
- Updates packaging model document (#1471)
- Fixes d2l book (https://d2l.djl.ai/) image display issue (#183)
- Translates d2l Chinese book (https://d2l-zh.djl.ai/) chapter 9 to chapter 14 to Chinese
Breaking change
- N/A
Bug Fixes
- Fixes several ci issues on Github Actions (#1358, #1362, #1363, #1364)
- Fixes crash when IValue list input is empty (#1440)
- Fixes NullPointException bug in getContextClassLoader() (#1445)
- Fixes model name detection issue in loading model from jar (#1446)
- Fixes unset numEmbeddings in TrainableWordEmbedding.java (#1450)
- Fixes protobuf-java CVEs
Contributors
This release is thanks to the following contributors:
- Andrey Zakharov (@zakhio)
- daofaziran1 (@daofaziran1)
- enpasos (@enpasos)
- Frank Liu (@frankfliu)
- Hampus Londögård (@Lundez)
- Jake Lee (@stu1130)
- James Zow(@jzow)
- Qing Lan (@lanking520)
- Zach Kimberg (@zachgk)
- Viet Yen Nguyen (@nguyenvietyen)
New Contributors
- @Lundez made their first contribution in #1412
- @zakhio made their first contribution in #1436
- @daofaziran1 made their first contribution in #1450
Full Changelog
Contributors
zakhio, Lundez, and daofaziran1
Assets 2
DJL v0.14.0 release
DJL v0.14.0 updates the engines PyTorch to 1.9.1 and introduces several new features:
Key Features
- Upgrades PyTorch engine to 1.9.1
- Adds support for Neuron SDK 1.16.1
- Adds autoscale in djl-serving for AWS Inferentia model
- Introduces OpenCV extension to provide high performance image processing
- Adds support for older version of PyTorch engine, user now can use PyTorch 1.8.1 with latest DJL
- Adds support for precxx11 PyTorch native library in auto detection mode
- Adds AWS Inferentia support in djl-bench
- Adds support for TorchServe .mar format, user can deploy TorchServe model archive in djl-serving
Enhancement
- Introduces several new features in djl-serving:
- Adds autoscale feature for AWS Inferentia (#31)
- Creates SageMaker hosting compatible docker image for AWS Inferentia (#36)
- Adds auto detect number of neuron cores feature for AWS Inferentia (#34)
- Adds autoscale support for SageMaker style .tar.gz model (#35)
- Adds support to load torchserve model (#32)
- Adds support to pip installed dependency per model (#37)
- Adds custom environment variable support for python engine (#29)
- Adds nested folder support in model archive file (#38)
- Improves model status with model version support (#25)
- Adds model warn up feature for python engine. (#23)
- Adds WorkLoadManager.unregisterModel (#33)
- Adds testing tool to test python model locally (#22)
- Adds set python executable path for python engine (#21)
- Creates Workflow for ModelServing (#26)
- Adds OpenCV extension (#1331)
- Introduces several new features in djl-bench:
- Adds support for AWS Inferentia (#1329)
- Introduces several new features in Apache MXNet engine:
- Implements LayerNorm for Apache MXNet (#1342)
- Introduces several new features in PyTorch engine:
- Upgrades PyTorch to 1.9.1 (#1297)
- Implements padding to bert tokenizer (#1328)
- Makes pytorch-native-auto package optional (#1326)
- Adds support to use different version of PyTorch native library (#1323)
- Adds map_location support for load model from InputStream (#1314)
- Makes map_location optional (#1312)
- Introduces several new features in TensorFlow Lite engine:
- Makes tensor-native-auto package optional (#1301)
- Introduces several API improvements:
- Adds support for nested folder in model archive (#1349)
- Improves translator output error message (#1348)
- Improves Predictor API to support predict with device (#1346)
- Improves BufferedImageFactory.fromNDArray performance (#1339)
- Adds support for downloading .mar file (#1338)
- Adds debugging toString to Input and Output (#1327)
- Refactors BERT Translator and Tokenizer (#1318)
- Makes question answering model serving ready (#1311)
- Refactors minMaxWorkers from ModelInfo to WorkerPool (#30)
Documentation and examples
- Adds huggingface Inferentia serving example (#184)
- Adds AWS SageMaker hosting document
- Adds python hybrid engine demo (#182)
- Improves DJL examples project gradle build script (#1344)
Breaking change
- PyTorch 1.9.1 no longer supports Amazon Linux 2, AL2 user has to use pytorch-native-cpu-precxx11
- Image.Type is removed and Image.duplicate() function no longer take Image.Type as input
- Image.getSubimage() is renamed to Image.getSubImage()
- PaddlePaddle model loading may break due to prefix changes.
Bug Fixes
- Fixes 2nd inference throw exception bug (#1351)
- Fixes calculation for SigmoidBinaryCrossEntropyLoss from sigmoid (#1345)
- Fixes jar model url download bug (#1336)
- Fixes memory in Trainer.checkGradients (#1319)
- Fixes NDManager is closed bug (#1308)
- Fixes PyTorch GPU model loading issue (#1302)
- Fixes MXNet EngineException message (#1300)
- Fixes python resnet18 demo model GPU bug (#24)
- Fixes python engine get_as_bytes() bug (#20)
Contributors
This release is thanks to the following contributors:
- Frank Liu (@frankfliu)
- Jake Lee (@stu1130)
- enpasos (@enpasos)
- Qing Lan (@lanking520)
- Zach Kimberg (@zachgk)
DJL v0.13.0 release
DJL v0.13.0 brings the new TensorRT and Python engines, and updates the engines PyTorch to 1.9.0, ONNXRuntime to 1.9.0, PaddlePaddle to 2.0.2, and introduces several new features:
Key Features
- Introduces TensorRT engine
- Introduces Python engine which allows you to run Python scripts with DJL
- Upgrades PyTorch engine to 1.9.0 with CUDA 11.1 support
- Upgrades ONNXRuntime engine to 1.9.0 with UINT8 support
- Upgrades PaddlePaddle engine to 2.0.2
- Introduces the djl-bench snap package:
sudo snap install djlbench --classic - Introduces dynamic batch feature for djl-serving (#1154)
- DJL serving becomes a standalone repository: https://github.com/deepjavalibrary/djl-serving (#1170)
- Allows load ModelZoo model using url (#1120)
- Support .npy and .npz file format (#1131)
- djl-serving is available in dockerhub
- Publishes d2l book Chinese translation preview (chapter 1-5) : https://d2l-zh.djl.ai
Enhancement
- Introduces several new features in djl-serving:
- Improves djl-serving API to make it easy to get HTTP headers (#1134)
- Loads models on all GPUs at startup for djl-serving (#1132)
- Enables asynchronous logging for djl-serving
- Makes djl-serving access log in separate log file (#1150)
- Adds configuration to support number worker threads for GPU inference (#1153)
- Improves auto-scale algorithm for djl-serving (#1149)
- Introduces several new features in djl-bench:
- Adds a command line option to djl-bench to generate NDList file (#1155)
- Adds warmup to benchmark (#1152)
- Improves djll-bench to support djl:// urls (#1146)
- Adds support to benchmark on multiple GPUs (#1144)
- Adds support to benchmark onnx on GPU machines (#1148)
- Adds support to benchmark TensorRT models (#1257)
- Adds support to benchmark Python models (#1267)
- Introduces several new features in PyTorch engine:
- Supports PyTorch custom input data type with IValue (#1208)
- Introduces several new features in OnnxRuntime:
- Adds UINT8 support for OnnxRuntime (#1271)
- Introduces several new features in PaddlePaddle:
- Introduces several API improvements:
- Adds missing NDList.get(String) API (#1194)
- Adds support to directly load models from a TFHub url (#1231)
- Improves repository API to support passing argument in the URL query string (#1139)
- Avoids loading the default engine if it is not being used (#1136)
- Improves IO by adding a buffer to read/write (#1135)
- Improves NDArray.toString() debug mode performance (#1142)
- Makes GPU device detection engine specific to avoid confusion when using multiple engines (#1138)
Documentation and examples
- Adds Style Transfer example with CycleGAN (#1180)
Breaking change
- Removes support for Apache MXNet 1.6.0
- Deprecates Device.getDevices() API - Use Engine.getDevices() instead
- Renames SimpleVocabulary to DefaultVocabulary
Bug Fixes
- Fixes broken link in documents
- Fixes TensorFlow NDArray was created on CPU instead of GPU bug (#1279)
- Fixes default image processing pipeline (#1268)
- Fixed XGBoost NDArray multiple read bug (#1239)
- Fixes platform matching bug (#1167)
- Fixes NullPointerException in NDArray.toString() (#1157)
- Fixes PaddlePaddle crash due to GC (#1162)
- Fixes NDArrayAdapter.getSparseFormat() unsupported bug (#1151)
- Fixes mixed device issue in multiple engine use case (#1123)
- Fixes handle duplicate plugin issue for djl-serving (#1108)
- Fixes XGBoost NDArray creation bug (#1109)
- Fixes runtime exception running benchmark arm machine(#1107)
- Fixes unregister model regression (#1101)
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi (@aksrajvanshi)
- Aziz Zayed (@AzizZayed)
- Elchanan Haas (@ElchananHaas)
- Erik Bamberg (@ebamberg)
- Frank Liu (@frankfliu)
- Jake Lee (@stu1130)
- Kimi MA (@kimim)
- Paul Greyson
- Qing Lan (@lanking520)
- Raymond Liu (@raymondkhliu)
- Sindhu Somasundaram (@sindhuvahinis)
- Zach Kimberg (@zachgk)
DJL v0.12.0 release
DJL v0.12.0 added GPU support to PaddlePaddle and ONNXRuntime, and introduces several new features:
Key Features
- Updates PaddlePaddle engine with GPU support.
- Updates ONNXRuntime engine with GPU support.
- Upgrades ONNXRuntime engine to 1.8.0.
- Upgrades XGBoost engine to 1.4.1.
- Introduces AWS Inferentia support, see our example for detail.
- Adds FLOAT16 datatype support in NDArray.
- Support UTF16 surrogate characters in NLP tokenization.
- Makes benchmark as a standalone tool.
- Releases djl-serving docker image to docker hub.
Enhancement
- DJL Benchmark now can benchmark any datatype as input.
- Makes Grayscale image processing match openCV’s behavior (#965)
- Improves PyTorch engine to load extra shared library for custom operators (#983)
- Improves djl-serving REST API to support load model on specified engine (#977)
- Improves djl-serving to support load multiple version of a model on the same endpoint (#1052)
- Improves djl-serving to support auto-scale workers based on traffic (#986)
- Implements several operators:
- Introduces several API improvements
Documentation and examples
- Adds Low cost inference with AWS Inferentia demo.
- Adds BigGAN demo in examples (#1038)
- Adds Super-resolution demo in examples (#1049)
Breaking change
- Direct access ModelZoo ModelLoader is no longer supported, use Criteria API instead.
- Deprecates ModelZoo.loadModel() API in favor of using Criteria.loadModel().
Bug Fixes
- Fixes missing softmax in action_recognition model zoo model (#969)
- Fixes saveModel NPE bug (#989)
- Fixes NPE bug in block.toString() function (#1076)
- Adds back String tensor support to TensorFlow engine (lost in 0.11.0 during refactor) (#1040)
- Sets ai.djl.pytorch.num_interop_threads default value for djl-serving (#1059)
Known issues
- The TensorFlow engine has a known memory leak issue due to the JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. You have to manually include javacpp 1.5.6-SNAPSHOT to avoid the memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more details.
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
- Aziz Zayed(@AzizZayed (https://github.com/AzizZayed))
- Erik Bamberg(@ebamberg (https://github.com/ebamberg))
- Frank Liu(@frankfliu (https://github.com/frankfliu))
- Hodovo(@hodovo (https://github.com/Hodovo))
- Jake Lee(@stu1130 (https://github.com/stu1130))
- Qing Lan(@lanking520 (https://github.com/lanking520))
- Tibor Mezei (@zemei (https://github.com/zemei))
- Zach Kimberg(@zachgk (https://github.com/zachgk))
DJL v0.11.0 release note
DJL v0.11.0 brings the new engines XGBoost 1.3.1, updates PyTorch to 1.8.1, TensorFlow to 2.4.1, Apache MXNet 1.8.0, PaddlePaddle to 2.0.2 and introduces several new features:
Key Features
- Supports XGBoost 1.3.1 engine inference: now you can run prediction using models trained in XGBoost.
- Upgrades PyTorch to 1.8.1 with CUDA 11.1 support.
- Upgrades TensorFlow to 2.4.1 with CUDA 11.0 support.
- Upgrades Apache MXNet to 1.8.0 with CUDA 11.0 support.
- Upgrades PaddlePaddle to 2.0.2.
- Upgrades SentencePiece to 0.1.95.
- Introduces the djl-serving brew package: now you can install djl-serving with
brew install djl-serving. - Introduces the djl-serving plugins.
- Introduces Amazon Elastic Inference support.
Enhancement
- Improves TensorFlow performance by reducing GC and fixed memory leaking issue (#892)
- djl-serving now can run all the engines out-of-box (#886)
- Improves DJL training by using multi-threading on each GPU (#743)
- Implements several operators:
- Adds setGraphExecutorOptimize option for PyTorch engine. (#904)
- Introduces String tensor support for ONNXRuntime (#724)
- Introduces several API improvements
- Introduces model searching feature in djl central (#799)
Documentation and examples
- Introduces DJL tutorials - How to load model on DJL Youtube Channel
- Adds the PaddlePaddle load model documentation (#811)
- Adds the documentations for profiler (#722)
- Adds face detection and face recognition examples (#814)
- Adds model training visualization demo using Vue
Breaking change
- Renames CheckpointsTrainingListener to SaveModelTrainingListener (#686)
- Removes erroneous random forest application (#726)
- Deletes DataManager class (#691)
- Classes under ai.djl.basicdataset packages has been moved into each sub-packages.
Bug Fixes
- Fixes BufferOverflowException when handling handling subimage (#866)
- Fixes ONNXRuntime 2nd engine dependency from IrisTranslator (#853)
- Fixes sequenceMask error when n dimension is 2 (#828)
- Fixes TCP port range buf in djl-serving (#773)
- Fixes one array case for concat operator (#739)
- Fixes non-zero operator for PyTorch (#704)
Known issues
- TensorFlow engine has known memory leak issue due to JavaCPP dependency. The memory leak issue has been fixed in javacpp 1.5.6-SNAPSHOT. User has to manually include javacpp 1.5.6-SNAPSHOT to avoid memory leak. See: https://github.com/deepjavalibrary/djl/tree/master/tensorflow/tensorflow-engine#installation for more detail.
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi(@aksrajvanshi (https://github.com/ghost))
- Anthony Feenster(@anfee1 (https://github.com/anfee1))
- Calvin(@mymagicpower (https://github.com/mymagicpower))
- enpasos(@enpasos (https://github.com/enpasos))
- Erik Bamberg(@ebamberg (https://github.com/ebamberg))
- Frank Liu(@frankfliu (https://github.com/frankfliu))
- G Goswami(@goswamig (https://github.com/goswamig))
- Hugo Miguel Ferreira(@hmf (https://github.com/hmf))
- Hodovo(@hodovo (https://github.com/Hodovo))
- Jake Lee(@stu1130 (https://github.com/stu1130))
- Lai Wei(@roywei (https://github.com/roywei))
- Qing Lan(@lanking520 (https://github.com/lanking520))
- Marcos(@markbookk (https://github.com/markbookk))
- Stan Kirdey(@skirdey (https://github.com/skirdey))
- Zach Kimberg(@zachgk (https://github.com/zachgk))
- 付颖志 (@fuyz (https://github.com/fuyz))
- 石晓伟(@Shixiaowei02 (https://github.com/Shixiaowei02))
DJL v0.10.0 Release
DJL v0.10.0 brings the new engines PaddlePaddle 2.0 and TFLite 2.4.1, updates PyTorch to 1.7.1, and introduces several new features:
Key Features
-
Supports PaddlePaddle 2.0 engine inference: now you can run prediction using models trained in PaddlePaddle.
-
Introduces the PaddlePaddle Model Zoo with new models. Please see examples for how to run them.
-
Upgrades TFLite engine to v2.4.1. You can convert TensorFlow SavedModel to TFLite using this converter.
-
Introduces DJL Central to easily browse and view models available in DJL’s ModelZoo.
-
Introduces generic Bert Model in DJL (#105)
-
Upgrades PyTorch to 1.7.1
Enhancement
- Enables listing input and output classes in ModelZoo lookup (#624)
- Improves PyTorch performance by using PyTorch index over engine agnostic solution (#638)
- Introduces various fixes and improvements for MultiThreadedBenchmark (#617)
- Makes the default engine deterministic when multiple engines are in dependencies(#603)
- Adds norm operator (similar to numpy.linalg.norm.html) (#579)
- Refactors the DJL Trackers to use builder patterns (#562)
- Adds the NDArray stopGradient and scaleGradient functions (#548)
- Model Serving now supports scaling up and down (#510)
Documentation and examples
- Introduces DJL 101 Video Series on DJL Youtube Channel
- Adds the documentation for Applications (#673)
- Adds an introduction for Engines (#660)
- Adds documents for DJL community and forums (#646)
- Adds documents on community leaders (#572)
- Adds more purpose to the block tutorial and miscellaneous docs (#607)
- Adds a DJL Paddle OCR Example (#568)
- Adds a TensorFlow amazon review Jupyter notebook example
Breaking change
- Renames DJL-Easy to DJL-Zero (#519)
- Makes RNN operators generic across engines (#554)
- Renames CheckpointsTrainingListener to SaveModelTrainingListener (#573)
- Makes Initialization optional in training (#533)
- Makes
SoftmaxCrossEntropyLoss'sfromLogitflag mean inputs are un-normalized (#639) - Refactors the Vocabulary builder API
- Refactors the SymbolBlock with AbstractSymbolBlock (#491)
Bug Fixes
- Fixes the benchmark rss value #656
- Fixes the recurrent block memory leak and the output shape calculation (#556)
- Fixes the NDArray slice size (#550)
- Fixes #493: verify-java plugin charset bug (#496)
- Fixes #484: support arbitrary URL scheme for repository
- Fixes AbstractBlock inputNames and the inputShapes mismatch bug
Known issues
- The training tests fail on GPU and Windows CPU if 3 engines(MXNet, PyTorch, TensorFlow) are loaded and run together
Contributors
This release is thanks to the following contributors:
- Akshay Rajvanshi(@aksrajvanshi )
- Anthony Feenster(@anfee1)
- Christoph Henkelmann(@chenkelmann)
- Frank Liu(@frankfliu)
- G Goswami(@goswamig)
- Jake Lee(@stu1130)
- James Zow(@Jzow)
- Lai Wei(@roywei)
- Marcos(@markbookk)
- Qing Lan(@lanking520)
- Tibor Mezei(@zemei)
- Zach Kimberg(@zachgk)
- Erik Bamberg(@ebamberg)
- Keerthan Vasist(@keerthanvasist)
- mpskowron(@mpskowron)