Skip to content

Releases: ARM-software/armnn

Release 24.02

22 Feb 13:27
@KevinARM KevinARM
v24.02
2431f76
Compare
Choose a tag to compare
Release 24.02 Latest
Latest

Summary

New Features

  • ArmNN to TOSA backend:
    • LeakyRelu Activation support added
    • Quantize support added
    • Maximum support added
    • Split support added
    • Resize Nearest Neighbour support added
  • GpuFsa Backend (Dynamic Fusion)
    • RESIZE/SCALE support added
    • CAST support added
    • POOL2d support added
    • SUB support added
    • ADD support added
    • DEPTHWISE CONVOLUTION 2D support added
    • CONVOLUTION 2D support added
  • Updated to Android NDK r26b.
  • Updated to TensorFlow 2.15.
  • Added optimization to remove reshape operators where possible to CL, Neon and Ref backends.

Bug Fixes

  • Removed implicit sign conversion which could cause compile errors.
  • Fixed memory leak which only happens during profiling and reference Resize workload's align corners is true.
  • Fixed build failures on C++ 14 compilers.
  • Fixed build tool errors when building for Android target.

Other Changes

  • Delegate Unit Tests are now only built for the backends which are being built.
  • Increased end to end testing for two layer and three layer MaxPool2d.
  • In ExecuteNetwork added support to serialize to dot graph for the Arm NN Delegates.

ABI/API Changes

No API breaking front-end changes have occurred during the implementation of 24.02.

No API breaking back-end changes have occurred during the implementation of 24.02.

Note: The minimum API level supported by Arm NN AAR file is 27.

Build Dependencies

Tools Supported Version
Git 2.17.1 or later
SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
Cmake 3.19.0 (Ubuntu) and 3.19.0 (Debian)
Tensorflow 2.15.0
Onnx 1.6.0
Flatbuffer 23.5.26
Protobuf 3.12.0
Android NDK r26b
mapbox/variant 1.2.0
cxxopts 3.1.1
doctest 2.4.6
fmt 8.3.0
ghc 1.3.2
half 1.12.0
mapbox/variant 1.2.0
stb 2.16
xxd 1.10
Assets 24

Release 23.11

23 Nov 12:08
@nikraj01 nikraj01
v23.11
3ba1ff4
Compare
Choose a tag to compare
Release 23.11

Summary

New Features

  • Add support for BROADCAST_TO layer in CpuRef, and remove it when it is followed by ElementWise layer.
  • Add an optimization that fuses Add+Mul+Add+(Optional Relu) layers in CpuAcc.
  • Add support for GELU activation layer in CpuRef, CpuAcc, GpuAcc.
  • Upgrade Arm NN to Tensorflow 2.14
  • Add Signed64 support
  • Add support for Signed64 data type in Cast layer
  • Add a script that evaluates the performance of a network
  • Add ReverseV2 CL and Neon Workloads

TfLite Parser

  • Add support for BROADCAST_TO layer.
  • Add support for GELU activation layer.
  • Updating TfLite parser to ignore VALIDATION: subgraphs

Arm NN Serializer/Deserializer:

  • Add support for GELU activation layer.

Bug Fixes

  • Fix UnidirectionalSequenceLstm
  • Fix weights checking when converting in Support Library
  • Fix unsafe Usages of Memcpy in Armnn
  • Fix for -Wno-sign-conversion in profiling test in gcc9
  • Fix ElementwiseBinary missing from NeonBackend activation fusion optimization
  • Fix Reshape and concat invalid results
  • Remove unnecessary Prelu restriction in quantization
  • Remove unnecessary Square Difference restriction in quantization

Other Changes

  • Update the Arm NN Execute Network app --help
  • Introduce clang-format scripts to ArmNN
  • Remove profiling detail for ConstTensorAsInputs Layers
  • Install missing profiling headers
  • Remove ASSERTs from deserializer code
  • Remove ASSERTs from armnnUtils code
  • Remove ASSERTs from shim code
  • Update documentation to correct C++ version: C++ 17
  • Removing explicit block on non constant bias in NEON CONV2D, allowing Arm Compute Library to handle this.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 23.11 that users should be aware of before upgrading. Due to these changes we have bumped our ARMNN_VERSION to 33.1.0 and our OPAQUE_DELEGATE_VERSION to 2.0.0, following Semantic Versioning guidelines.

Feature SHA Gerrit Review Resultant ABI/API changes
Add ArmNNSettings to Opaque Delegate 3e4b608 https://review.mlplatform.org/c/ml/armnn/+/10493 2 changes have occurred:
  • TfLiteArmnnOpaqueDelegateCreate function has a different signature: Previously: TfLiteOpaqueDelegate* TfLiteArmnnOpaqueDelegateCreate(const void* settings); Now: TfLiteOpaqueDelegate* TfLiteArmnnOpaqueDelegateCreate(armnnDelegate::DelegateOptions options);

  • Size of struct ArmnnDelegatePlugin has increased as a new private member has been added: armnnDelegate::DelegateOptions m_delegateOptions;

No API breaking back-end changes have occurred during the implementation of 23.11

TfLite Delegate

  • Add support for BROADCAST_TO layer to Classic and opaque delegate.
  • Add support for GELU activation layer to classic and opaque delegate.
  • Add ArmNNSettings parser function for Opaque Delegate.
  • Improve logging in the delegate

Bug Fixes

  • Reduce Sum uint8 failing. The fix was to only treat Reduce Prod Uint8 as a special case, as opposite of treating all reduce operations in uint8 as special case (kTfLiteAffineQuantization → kTfLiteNoQuantization)
  • Fix Issue with delegate supporting FP16 models
  • Delegate Test Suite: Fix reshape floating point exception
  • Delegate Test Suite: Fix default scale/offset issue
  • Delegate Test Suite: Fix ElementWise isnan assert
  • Delegate Test Suite: Fix Unspecified dimension while using ShapeInferenceMethod::ValidateOnly
  • Delegate Test Suite: Fix QuantizePerChannel tests
  • Delegate Test Suite: Fix Gather and GatherNd Tests in CpuRef

PyArmNN

  • Update requests version in PyArm NN
  • Bump Pillow version from 9.3.0 to 10.0.1

Build Dependencies

Tools Supported Version
Git 2.17.1 or later
SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
Cmake 3.19.0 (Ubuntu) and 3.19.0 (Debian)
Tensorflow 2.14.0
Onnx 1.6.0
Flatbuffer 23.5.26
Protobuf 3.12.0
Android NDK r25
mapbox/variant 1.2.0
cxxopts 3.1.1
doctest 2.4.6
fmt 8.3.0
ghc 1.3.2
half 1.12.0
mapbox/variant 1.2.0
stb 2.16
xxd 1.10
Assets 24

Release 23.08

24 Aug 14:03
@nikraj01 nikraj01
v23.08
ef0f3e7
Compare
Choose a tag to compare
Release 23.08

Summary

New Features

  • Added support for tile operator in CpuRef, CpuAcc, GpuAcc.
  • Added support for reverse_v2 operator in CpuRef.
  • Added pow and squared_difference as ElementWiseBinary layers in CpuRef, CpuAcc, and GpuAcc.
  • Added squared_difference, power and ceil to TypeUtils.hpp.
  • Enabled dynamic / non-constant bias for:
    • Fully-Connected layers in CpuAcc and GpuAcc
    • 3-D Convolutional layers in CpuAcc and GpuAcc
    • Depthwise Convolutional layers in GpuAcc
  • Added DataType to .dot files for constant layers.
  • Added BinaryElementwiseOperation to .dot files.
  • Added a FileComparisonExecutor to ExecuteNetwork.
  • Added an optional TensorInfo to InputSlot.
  • Added 3D tensors to batch_to_space and space_to_batch for CpuAcc and GpuAcc.
  • Added check for half-precision floating-point infinity values and backend support (FP16).
  • Added backend optimisations to remove reshape layers where possible.
  • Added data layout to tensors in NeonStridedSliceWorkload.
  • Added names to workloads.
  • Enabled slice end-to-end tests in all backends and Signed32 in CpuRef.
  • Added axis to ViewsDescriptor.
  • Refactored ElementBinaryOps to use ElementBinaryLayer.

TfLite Parser

  • Added reverse_v2 support to TFLite Parser.
  • Added tile to TFLite Parser.
  • Added square as mul in the TFLite Parser.
  • Check for options != null before adding fused activation in TFLite Parser.
  • Fixed segfault with some models in the TFLite Parser.

Arm NN Serializer/Deserializer:

  • Added tile to Serialiser/Deserialiser.
  • Added reverse_v2 to Serialiser/Deserialiser.

Support library

  • Added reverse_v2 to Support Library.
  • Added tile to Support Library.
  • Added cache-size check to Support Library.

Bug Fixes

  • Fixed incorrect validation of unidirectional_sequence_lstm on CL and Neon.
  • Fixed issue with ExecuteNetwork when running with TFLite Executor.
  • Replaced asserts with exceptions in Gather reference workload.
  • Introduced fix to explicitly state the correct header to be included (following prior deprecation warning).
  • Fixed XML parsing error in Arm NN Doxygen.
  • Fixed -Werror=unused-result error.
  • Introduced fix for ExecuteNetwork where --output-network-details-only was not working with -T delegate flag.
  • Introduced fix for duplicate definitions in cross-compilation build.
  • Fixed incorrect Concat permutation parameters in Support Library.
  • Removed unnecessary warnings for certain models.
  • Introduced fix to allow SplitterLayer to use overridden TensorInfos correctly.
  • Introduced fix for some cases where the use of sub-tensors was causing an error.
  • Fixed read memory access caused by missing printf arguments.
  • Introduced fix for failing dynamic backend build.
  • Fixed issue where the dimension's specificity didn't match the number of dimensions.
  • Fixed ambiguous method name in BackendHelper.
  • Introduced fix for segmentation fault when an input was directly connected to an output.
  • Fixed uninitialised variable error found during static analysis.
  • Fixed fault in ExecuteNetwork when a model file was passed without an extension.
  • Fixed GitHub issue where search bar was not working in Doxygen documentation.

Other Changes

  • Replaced use of std::filesystem with ghc::filesystem.
  • Refactored ConnectedToSplitterWithMoreThan4Dims function to a more generally useful ConnectedToLayerType function.
  • Customised Doxygen output.
  • Removal of deprecated code due to be removed in 23.08 or earlier:
    • INetworkProperties
    • SubgraphView
    • ILayerSupport
    • WorkloadFactory
  • Updated documentation with new operators in 23.08.
  • Audited the use of armnn_assert.

Known Issues

  • Intermittent issue on Dma Buf memory import on GPU. This is fix in Mali Driver r30p0.
  • There might be performance regressions against v20.08 in Inception v3 using int8 data types on Arm Mali-G77 GPUs. Currently under investigation.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 23.08 that users should be aware of before upgrading. Due to these changes we have bumped our ARMNN_VERSION to 33.0.0, following Semantic Versioning guidelines.

Feature SHA Gerrit Review Resultant ABI/API changes
Removal of Reshape 4cc341c https://review.mlplatform.org/c/ml/armnn/+/9885 4 additional virtual methods added to class IInputSlot:
SetTensorInfo ( TensorInfo ),
GetTensorInfo ( ) const,
IsTensorInfoSet ( ) const,
IsTensorInfoOverridden ( ) const
Front end and reference implementation for TILE 79a06a5 https://review.mlplatform.org/c/ml/armnn/+/9920 LayerType enum has had the LastLayer member value changed from 72 to 74 The member Tile with value 74 has been added
Remove deprecated code 09e4d05 https://review.mlplatform.org/c/ml/armnn/+/9266 2 functions have been removed from the BatchMatMulDescriptor class:
  • BatchMatMulDescriptor::GetAxesNotMul ( struct BatchMatMulDescriptor const& desc, TensorShape const& inputXShape, TensorShape const& inputYShape ) [static]
  • BatchMatMulDescriptor::GetAxesToMul ( struct BatchMatMulDescriptor const& desc, TensorShape const& tensorXShape, TensorShape const& tensorYShape ) [static]
Remove deprecated code (INetworkProperties) b179382 https://review.mlplatform.org/c/ml/armnn/+/10001 INetworkProperties structure has had the following fields removed: m_ExportEnabled m_ImportEnabled
Remove deprecated code (ILayerSupport) 6627703 https://review.mlplatform.org/c/ml/armnn/+/10005 In ArmNN individual virtual IsXXXSupported() functions in the ILayerSupport class have been removed. This functionality has been replaced by a more ABI compliant model whereby an IsLayerSupported() function now accepts a LayerType argument.
In ArmNNTestUtils, removal of 4 virtual methods from class MockLayerSupport:
  • IsAdditionSupported ( TensorInfo const&, TensorInfo const&, TensorInfo const&, Optional<std::__cxx11::basic_string&> ) const
  • IsConvolution2dSupported ( TensorInfo const&, TensorInfo const&, struct Convolution2dDescriptor const&, TensorInfo const&, Optionalconst&, Optional<std::__cxx11::basic_string&> ) const
  • IsInputSupported ( TensorInfo const&, Optional<std::__cxx11::basic_string&> ) const
  • IsOutputSupported ( TensorInfo const&, Optional<std::__cxx11::basic_string&> ) const
Remove deprecated code (WorkloadFactory) 7894ef9 https://review.mlplatform.org/c/ml/armnn/+/10006 In ArmNNTestUtils the MockWorkloadFactory class has had the following virtual method removed: CreateInput ( InputQueueDescriptor const&, struct WorkloadInfo const& ) const
Added Axis to ViewsDescriptor fca5916 https://review.mlplatform.org/c/ml/armnn/+/10073 The size of thte ViewsDescriptor structure has changed from 48 bytes to 56 bytes Field m_IsAxisSet has been added

The following back-end API changes have occurred during the implementation of 23.08 that users should be aware of before upgrading.

Feature SHA Gerrit Review Resultant ABI/API changes
Add names to workloads 7cbe781 https://review.mlplatform.org/c/ml/armnn/+/9983 Pure virtual method GetName ( ) const has been added to IWorkload class. Size of CopyMemGenericWorkload class has been changed from 152 bytes to 184 bytes.
Remove deprecated code (SubgraphView) 0f3e9a0 https://review.mlplatform.org/c/ml/armnn/+/10009 Removed the following methods from SubgraphView:
GetInputSlot, GetIputSlots, GetLayers, GetOutputSlot, GetOutputtSlots.
Resulted in a change to return types for the following methods in SubgraphView as follows:
begin() returns SubgraphView::IConnectableLayerIterator,
begin() const returns SubgraphView::ConstIConnectableIterator,
cbegin() const returns, SubgraphView::ConstIConnectableIterator,
cend() const returns SubgraphView::ConstIConnectableIterator ,
end () returns SubgraphView::IConnectableLayerIterator,
end() const returns SubgraphView::ConstIConnectableIterator
Remove deprecated code (ILayerSupport) a504834 https://review.mlplatform.org/c/ml/armnn/+/10071 Made IWorkloadFactory::CreateWorkload a pure virtual function to force client to write its own implementation.
Fix coverity error on variable initialize b9b9792 https://review.mlplatform.org/c/ml/armnn/+/10075 Adjusted sequence of variable initialization in struct WorkloadInfo:
m_Name
m_WeightsTensorInfo
m_BiasTensorInfo
m_ConvolutionMethod

TfLite Delegate

  • Extended support for 3D tensors (`bat...
Read more
Assets 18

Release 23.05

22 May 09:27
@nikraj01 nikraj01
v23.05
49f609d
Compare
Choose a tag to compare
Release 23.05

Summary

New Features

  • Added support for dynamic weights in CpuAcc and GpuAcc for FullyConnected workloads.
  • Added support for Crops in CpuAcc and GpuAcc BatchToSpaceND workloads.
  • Added support for int8 and changed the Compute Library kernel used for Fp32 in CpuAcc and GpuAcc Batch MatMul workloads.
  • Added Opaque TfLite Delegate which provides the same operator coverage as the existing/classic TfLite Delegate. More information can be found in the TfLite Delegate section below

TfLite Parser

  • Added support for CEIL and SPACE_TO_DEPTH operators.
  • Fixed bug where calculated output shape wasn't being recorded in ParseSqueeze.
  • Fixed segfault in ParseTransposeConv2d when output_shape is not constant.
  • Fixed bug where negative axis was being read incorrectly in ParseMean.
  • Calculate explicit padding for Transpose Convolution using output shape, if specified.

ONNX Parser

  • Added support for dimensions > 2 to MatMul/FullyConnected.

Arm NN Serializer/Deserializer:

  • Added support for CEIL.

Bug Fixes

  • Fixed compare-output output feature in ExecuteNetwork.
  • Fixed gcc 13 compiler errors.

Other Changes

  • Added ElementwiseBinaryLayer to replace Add, Div, Sub, Maximum, Mul and Minimum layers.
  • Updated build Android NDK guide (BuildGuideAndroidNDK.md).
  • Set default quantization parameter scale to 1.0, instead of 0.0.
  • Fp16ToFp32 and Fp32ToFp16 convert workloads now use arm_compute::NECast in CpuAcc backend, when available. This should in general be faster.
  • Added Opaque TfLite Delegate build option to the build-tool.

Known Issues

  • Intermittent issue on Dma Buf memory import on GPU. This is fix in Mali Driver r30p0.
  • There might be performance regressions against v20.08 in Inception v3 using int8 data types on Arm Mali-G77 GPUs. Currently under investigation.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 23.05 that users should be aware of before upgrading. Note: No ABI breaking change occured in ArmNN Core (libarmnn.so) and so the Major version has not changed, only a bump in minor version (32.0.0 → 32.1.0).

Feature SHA Gerrit Review Resultant ABI/API changes
Implement Pimpl Idiom for Delegate Options 1bae865 https://review.mlplatform.org/c/ml/armnn/+/9358 Size of class DelegateOptions has been changed from 488 bytes to 8 bytes. Layout of parameter's stack of several functions has been changed and therefore parameters at higher positions in the stack may be incorrectly initialized by applications. Size of class Delegate has been changed from 552 bytes to 72 bytes. Size of field m_Options has been changed from 488 bytes to 8 bytes. The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications.
Implement Pimpl Idiom for OptimizerOptions c5ee0d7 https://review.mlplatform.org/c/ml/armnn/+/9369 The following functions have been changed to now accept OptimizerOptionsOpaque argument instead of the unstable OptimizerOptions.
Note: OptimizerOptionsOpaque will accept an OptimizerOptions to it's constructor so this is not an API break, only an ABI break.
DelegateOptions::DelegateOptions [C1] ( enum armnn::Compute computeDevice, struct armnn::OptimizerOptions const& optimizerOptions,armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int,armnn::ITensorHandle*)> >const& func)
DelegateOptions::DelegateOptions [C2] ( enum armnn::Compute computeDevice, struct armnn::OptimizerOptions const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::DelegateOptions [C1] ( std::vectorarmnn::BackendIdconst& backends, struct armnn::OptimizerOptions const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::DelegateOptions [C2] ( std::vectorarmnn::BackendIdconst& backends, struct armnn::OptimizerOptions const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::SetOptimizerOptions ( struct armnn::OptimizerOptions const& optimizerOptions)
Replacement functions:
DelegateOptions::DelegateOptions [C1] ( enum armnn::Compute computeDevice, armnn::OptimizerOptionsOpaque const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel,armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::DelegateOptions [C2] ( enum armnn::Compute computeDevice, armnn::OptimizerOptionsOpaque const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::DelegateOptions [C1] ( std::vectorarmnn::BackendIdconst& backends, armnn::OptimizerOptionsOpaque const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::DelegateOptions [C2] ( std::vectorarmnn::BackendIdconst& backends, armnn::OptimizerOptionsOpaque const& optimizerOptions, armnn::Optionalarmnn::LogSeverityconst& logSeverityLevel, armnn::Optional<std::function<void(arm::pipe::ProfilingGuid, unsigned int, armnn::ITensorHandle*)> >const& func)
DelegateOptions::SetOptimizerOptions ( armnn::OptimizerOptionsOpaque const& optimizerOptions )
Add constant version of IConnectableLayer::GetConstantTensorsByRef aeec3ce https://review.mlplatform.org/c/ml/armnn/+/9196 For class IConnectableLayer a pure virtual method GetConstantTensorsByRef ( ) const has been added. Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method.
Note: The pure virtual function was added at the end so will not result in an ABI break. However you should usually not add new virtual functions for any reason, even to leaf classes, if the class is intended to remain binary compatible on Windows. Doing so may reorder existing virtual functions and break binary compatibility. As we do not target support for Windows we are not going to consider this an ABI break for ArmNN Core.

The following back-end API changes have occurred during the implementation of 23.05 that users should be aware of before upgrading.

Feature SHA Gerrit Review Resultant ABI/API changes
Remove GetGraph and include of Graph.hpp header from public header c1c5f2a https://review.mlplatform.org/c/ml/armnn/+/9351 Size of class OptimizationViews has been changed from 360 bytes to 88 bytes. Field m_Graph has been removed from this type. The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications.

TfLite Delegate

  • The existing TfLite Delegate has now been renamed to "Classic" TfLite Delegate to accommodate the new Opaque TfLite Delegate. There has been a file restructure because of this.
  • The Opaque TfLite Delegate provides the same operator coverage as the existing/classic TfLite Delegate. A list of these supported operators can be found in the TfLite Delegate section of the documentation

New features

  • Added support for CEIL, EXPAND_DIMS and SQUEEZE operators.

Bug Fixes

  • Fixed layer support for Comparison, ElementWiseBinary and LogicalBinary operators, by expanding the TensorShape before verifying support, if required.
  • Fixed handling of a negative size value in the Slice operator.
  • Calculate explicit padding for Transpose Convolution using output shape, if specified.

Build Dependencies

Tools Supported Version
Git 2.17.1 or later
SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
Cmake 3.5.1 (Ubuntu) and 3.7.2 (Debian)
Tensorflow 2.12.0 (SHA 6f692f73cb2043b4a0b0446539cd8c15b3dd9220)
Onnx 1.6.0
Flatbuffer 2.0.6
Protobuf 3.12.0
Android NDK r25
mapbox/variant 1.2.0
cxxopts 3.1.1 (SHA eb787304d67ec22f7c3a184ee8b4c481d04357fd)
doctest 2.4.6
fmt 7.0.1
ghc 1.3.2
half 1.12.0
mapbox/variant 1.2.0
stb 2.16
xxd 1.10
Assets 15

Release 23.02

09 Mar 17:13
@nikraj01 nikraj01
v23.02
f233be4
Compare
Choose a tag to compare
Release 23.02

New Features

  • Arm NN TOSA Backend
    • Added Concatenation support to TOSA Reference Backend.
    • Added Constant layer support to TOSA Reference Backend.
    • Added Convolution 2D support to TOSA Reference Backend.
    • Added Pooling2d support to TOSA Reference Backend.
    • Added Reshape support to TOSA Reference Backend.
    • Added RSqrt support to TOSA Reference Backend.
    • Added Slice support to TOSA Reference Backend.
    • Added Transpose Convolution 2D support to TOSA Reference Backend.
    • Added Subtraction and Multiplication support to TOSA Reference Backend.
  • Added support for GpuAcc BatchMatMul with FP32.
  • Extend BatchMatMul support for 4D tensors in GpuAcc.

ONNX Parser

  • Provide a CreateNetworkFromBinary method for the ONNX parser.

TfLite Parser:

  • Fixed issue in ParseReshape where the targetShape wasn't always calculated correctly.
  • Fixed issue in ParseFullyConnected where the wrong name was used for the ReshapeLayer.
  • Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly.

Bug Fixes

  • Bug fixed on ExecuteNetwork when input names where not given, input files were not used.
  • Bug Fixed on delegate Profiling in ExecuteNetwork with multiple iterations.
  • Bug Fixed for CpuAcc and GpuAcc. BuildArmComputePermutationVector() function needed to be rewritten to account for all possible permutation vectors.
  • Fixed an ExecuteNetwork unhandled exception when using option --import-inputs-if-aligned.
  • Fixed Arm NNAPI Support Library to fail gracefully if device is unavailable.
  • Fixed edge cases where some permute vectors for Arm Compute were not converted correctly.
  • Fixed bug where GPU backend options were not being correctly passed by our delegate.
  • Fixed bug when converting Constants with Per-Axis Quantization.
  • Fixed bug where call on SubstituteSubgraph on working copy of subgraph in Optimize fails.
  • Fixed segfault in ExecuteNetwork when no operator is supported by Arm NN.
  • Fixed bug for slot replacement during UpdateSubgraphViewSlotPointers.
  • Fixed bug for ExecuteNetwork using delegate when output is boolean from comparison layer.

Other Changes

  • Disabled BF16-Turbo-Mode and remove conversion layers.
  • Added Arm NN include directory into build-tool output.
  • Code improvement through removal of unused includes.
  • Optimization of IsLayerSupported to reduce calls to it.
  • Removed deprecated code due to be removed in 23.02.
  • Changed Arm NN Support LIbrary to use static libraries instead of object libraries.
  • Added option of static build of Execute Network.
  • Improved error handling when ExecuteNetwork creates a directory when -F option used.
  • Changed ArmNNExecutors to now share a single IRuntime, which allows ExecuteNetwork to create and run multiple Executors instead of one.
  • Added documentation relating to multithreading.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 23.02 that users should be aware of before upgrading.
.

Feature SHA Gerrit Review Resultant ABI/API changes
Optimize the calling of IsLayerSupported(). 5383767 https://review.mlplatform.org/c/ml/armnn/+/8742 In class IConnectableLayer: Pure virtual method SetBackendId (BackendId const&) has been added to this class. Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method. The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
When creating multiple Executors only the last one works fine 5446a4d https://review.mlplatform.org/c/ml/armnn/+/8997 In class Delegate: Size of this type has been changed from 688 bytes to 680 bytes. The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications. Type of field m_Runtime has been changed from armnn::IRuntimePtr (16 bytes) to armnn::IRuntime* (8 bytes). Size of the inclusive type has been changed
Fix incorrect last layer in Types.hpp 6701daf https://review.mlplatform.org/c/ml/armnn/+/8944 In enum LayerType: Value of member LastLayer has been changed from 66 to 71. Applications may execute a wrong branch of code in the library and therefore change the behavior.
Change to MemorySource to keep it usable as a bit mask 1cebf49 https://review.mlplatform.org/c/ml/armnn/+/9053 In enum MemorySource: Value of member Gralloc has been changed from 5 to 8. Applications may execute a wrong branch of code in the library and therefore change the behavior.

The following back-end API changes have occurred during the implementation of 23.02 that users should be aware of before upgrading.

Feature SHA Gerrit Review Resultant ABI/API changes
Remove deprecated code due to be removed in 23.02 ec67a0f https://review.mlplatform.org/c/ml/armnn/+/8319 In struct Convolution2dQueueDescriptor, DepthwiseConvolution2dQueueDescriptor and FullyConnectedQueueDescriptor: Field m_Bias has been removed from this type. Field m_Weight has been removed from this type. 1) Applications will access incorrect memory when attempting to access this field. 2) Size of the inclusive type has been changed. 3) The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications. In class BaseWorkload , BaseWorkload and BaseWorkload: 1) Size of the inclusive type has been changed. 2) Previous accesses of applications and library functions to this field and fields at higher positions of the structure definition may be broken. 3) The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications.
Return INetwork* not INetworkPtr& from OptimizationViews::GetINetwork() 5b2145c https://review.mlplatform.org/c/ml/armnn/+/8828 In OptimizationViews::GetNetwork(): Base type of return value has been changed from std::unique_ptr<INetwork, void()(INetwork)> to INetwork. Recompilation of a client program may be broken.
Allow working copy SubgraphView to get Original Slots 01f7269 https://review.mlplatform.org/c/ml/armnn/+/8918 In class SubgraphView: Base class std::enable_shared_from_this has been added. 1) Size of the class has been changed from 160 bytes to 176 bytes. 2) The memory layout in this class has been shifted by 16 bytes. 3) The class has only inline or auto-generated constructors which will be copied to applications at compile time and will allocate an older memory layout. Call of any exported method of this class may access a memory outside the allocated objects or inside the older memory structure and result in crash or incorrect behavior of applications. 4) The memory layout and size of subclasses will be changed.

TfLite Delegate

New features

  • Added support for Slice operator.
  • Made change to allow constant tensors as inputs for input data in the delegate.

Bug Fixes

  • Fixed delegate fallback during VisitNode so that an ArmNN exception is now caught and the process is handed over to TFLite.
  • Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly.
  • Fixed delegate fallback when fused activation is unsupported.
  • Fixed uncaught warnings treated as errors in delegate release build.

PyArm NN

  • Add installation instructions for prebuilt binaries.

Build Dependencies

Tools Supported Version
Git 2.17.1 or later
SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
Cmake 3.5.1 (Ubuntu) and 3.7.2 (Debian)
Tensorflow 2.10.0
Onnx 1.6.0
Flatbuffer 2.0.6
Protobuf 3.12.0
Android NDK r25
mapbox/variant 1.2.0
cxxopts SHA 12e496da3d486b87fa9df43edea65232ed852510
doctest 2.4.6
fmt 7.0.1
ghc 1.3.2
half 1.12.0
stb 2.16
xxd 1.10
Assets 12

Release 22.11.01

23 Jan 14:58
@nikraj01 nikraj01
v22.11.01
05509cb
Compare
Choose a tag to compare
Release 22.11.01

Summary

This is a patch release to fix an issue in the Arm Support Library encountered on Android phones where the OpenCL libraries could not be detected.

In this case the 22.11 release was detecting the issue and throwing an exception but the Tensorflow Lite runtime was expecting an error code so fallback to the runtime was failing.

In this release an error code is being returned when a misconfigured/missing OpenCL installation is encountered and the Tensorflow Lite runtime is taking over execution of the graph as expected.

This 22.11.01 release contains all the features of Arm NN 22.11 release. Please find release note for 22.11 here https://github.com/ARM-software/armnn/releases/tag/v22.11.

Assets 2

Release 22.11

28 Nov 16:08
@nikraj01 nikraj01
v22.11
c6fb7d1
Compare
Choose a tag to compare
Release 22.11

Summary

New Features

  • ArmNN to TOSA backend:
    • Added TOSA Mappings backbone structure with support for Addition operator (Float32).
    • Implemented simple TOSA Reference Backend skeleton.
    • Implemented TosaRefBackend::OptimizeSubgraphView.
    • Integrated TOSA Serialization Library into Arm NN.
    • Integrated TOSA Reference Model into Armn NN.
  • BATCH_MATMUL:
    • Added adjoint and transpose parameters to BATCH_MATMUL layer and CpuRef workload.
    • Added support for BATCH_MATMUL to Arm NN Support Library.
    • Added support for BATCH_MATMUL FP32 to CpuAcc.
    • Added BATCH_MATMUL end to end tests.
  • Updated to Android NDK r25.
  • Updated to TensorFlow 2.10 and Flatbuffers 2.0.6.

TfLite Parser

  • Added BATCH_MATMUL to TFLite Parser.
  • Fixed bug in TFLite Parser failing to prepare model due to unspecified size buffer data for SLICE operator.
  • In TFLite Parser we observed that in BATCH_MATMUL layer, when adjoint parameter was true, the mathematical calculation was transpose. So we linked adjoint from TFLite to transpose in ArmNN.
  • Added support for RESHAPE when output 'shape_signature' parameter contains a value of -1 in TFLite Parser.

ArmNN Serializer/Deserializer

  • Added support for BATCH_MATMUL to Serializer/Deserializer.

Bug Fixes

  • Fixed bug in SubgraphView::SubstituteSubgraph where IOutputSlots were incorrectly overridden.
  • Fixed bug in ExecuteNetwork when iterations and input files are not matching.
  • Updated SubgraphView Selector to give deterministic results.
  • Fixed bug in ArmNNExecutor where errors from LoadNetwork were being ignored in.
  • Fixed bug with debug mode not working correctly with Constant Tensors as Inputs.
  • Fixed incorrect kernel measurement in profiling output.
  • Fixed ExecuteNetwork for multiple outputs.
  • Make the AllowExpandedDims option work.
  • Fixed output format issue for int8 when using -w in ExecuteNetwork.

Other Changes

  • Added runtime options to Doxygen.
  • Added message deprecating the use of master branch. main branch is now used.
  • Removed deprecated code due to be removed in 22.08 as we cold not do this in 22.08.
  • Removed deprecated code due to be removed in 22.11.
  • Delayed the removal of deprecated weights and bias by one release.
  • Generalized get_compute_library.sh usage.
  • Use ARMNN_VERSION for Support Library version String.
  • Removed aarch32 build from build-tool.
  • Forward declare ILocalPacketHandlerSharedPtr in IRuntime.hpp
  • Use stricter file extension check in CreateParser.

Note: Following the upgrades to Tensorflow 2.10 and Flatbuffers 2.0.6 a compiler that supports C++17 is now required. This will prevent compilation on some older operating systems, e.g. Debian 9.

ABI/API Changes

The following front-end API changes have occurred during the implementation of 22.11 that users should be aware of before upgrading.
.

Feature SHA Gerrit Review Resultant ABI/API changes
Remove deprecated code 22.08 48f9d5d https://review.mlplatform.org/c/ml/armnn/+/8167 Removed Symbols: INetwork::AddConvolution2dLayer ( struct Convolution2dDescriptor const& convolution2dDescriptor, ConstTensor const& weights, Optionalconst& biases, char const* name ). INetwork::AddDepthwiseConvolution2dLayer ( struct DepthwiseConvolution2dDescriptor const& convolution2dDescriptor, ConstTensor const& weights, Optionalconst& biases, char const* name )
Implement simple TOSA Reference Backend skeleton ae8a6f5 https://review.mlplatform.org/c/ml/armnn/+/8082 In class MockWorkloadFactory the following has changed:
  • The relative position of virtual method CreateInput ( InputQueueDescriptor const&, struct WorkloadInfo const& ) const has been changed from 5 to 8.
  • The relative position of virtual method CreateWorkload ( enum LayerType, struct QueueDescriptor const&, struct WorkloadInfo const& ) const has been changed from 8 to 7.
  • The relative position of virtual method CreateTensorHandle ( TensorInfo const&, enum DataLayout, bool const ) const has been changed from 7 to 6.
  • The relative position of virtual method CreateTensorHandle ( TensorInfo const&, bool const ) const has been changed from 6 to 5.
  • The layout of v-table has been changed. Call of these virtual methods may result in crash or incorrect behavior of applications.
    		•
    Fix AllowExpandedDims option 16c76d5 https://review.mlplatform.org/c/ml/armnn/+/8419 Added Symbols:
  • INetwork::Create ( NetworkOptions const& networkOptions ) [static] INetwork::CreateRaw ( NetworkOptions const& networkOptions ) [static]
  • Removed Symbols: INetwork::Create ( NetworkOptions networkOptions ) [static] INetwork::CreateRaw ( NetworkOptions networkOptions ) [static]. Effectively the parameters list has been changed for the above functions. The name of the appropriate symbol for these functions on binary level has been changed. This may cause undefined reference linker error in old client applications.
  • struct OptimizerOptions: Field m_AllowExpandedDims has been added to this type. This field will not be initialized by old clients. NOTE: this field should be accessed only from the new library functions, otherwise it may result in crash or incorrect behavior of applications.
    		•
    Add functionality to print output tensors to file 7bbf565 https://review.mlplatform.org/c/ml/armnn/+/8421 struct OptimizerOptions: Field m_DebugToFile has been added at the middle position of this structural type. Layout of structure fields has been changed and therefore fields at higher positions of the structure definition may be incorrectly accessed by applications.

    The following back-end API changes have occurred during the implementation of 22.11 that users should be aware of before upgrading.

    Feature SHA Gerrit Review Resultant ABI/API changes
    Implement simple TOSA Reference Backend skeleton ae8a6f5 https://review.mlplatform.org/c/ml/armnn/+/8082 ILayerSupport.hpp Changed pure virtual function IsChannelShuffleSupported to virtual function.WorkloadFactory.hpp.The relative position of virtual function CreateInput(const InputQueueDescriptor& descriptor, const WorkloadInfo& info) const has been moved.The layout of v-table has been changed. Call of these virtual methods may result in crash or incorrect behavior of applications.
    Fix AllowExpandedDims option 16c76d5 https://review.mlplatform.org/c/ml/armnn/+/8419 const has been added for the constructor OptimizationViews(const NetworkOptions& networkOptions = {}) : m_INetwork(INetwork::Create(networkOptions)) As a result the layout of v-table has been changed. Calls of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    Remove deprecated code 22.08 d1628bf https://review.mlplatform.org/c/ml/armnn/+/8167 Removed in WorkloadData.hpp ResizeBilinearQueueDescriptor has been removed.

    TfLite Delegate

    New features

    • Added a no fallback mode to the TfLite Delegate. This should only be used for testing purposes.

    Build Dependencies

    Tools Supported Version
    Git 2.17.1 or later
    SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
    Cmake 3.5.1 (Ubuntu) and 3.7.2 (Debian)
    Tensorflow 2.10.0
    Onnx 1.6.0
    Flatbuffer 2.0.6
    Protobuf 3.12.0
    Android NDK r25
    mapbox/variant 1.2.0
    cxxopts SHA 12e496da3d486b87fa9df43edea65232ed852510
    doctest 2.4.6
    fmt 7.0.1
    ghc 1.3.2
    half 1.12.0
    stb 2.16
    xxd 1.10
    Assets 13

    Release 22.08

    25 Aug 15:43
    @nikraj01 nikraj01
    v22.08
    0ba0b2b
    Compare
    Choose a tag to compare
    Release 22.08

    Summary

    New Features

    • Add Arm NN Support Library.
      • The Arm NN Support Library for Android NNAPI is a shared library which has all the functionalities of existing HAL drivers for Android NNAPI.
      • It is available from Android S.
      • It focuses on update-ability of ML operators.
      • Guiide on how to build Arm NN Support Library is available armnn/shim/BuildGuideShimSupportLibrary.md.
      • SLTS (Support Library Test Suit) compliance.
    • Support for Batch MatMul in CpuRef.

    TfLite Parser

    • Added support for LOG.
    • Added support for SIN.

    ExecuteNetwork App Changes:

    • Refactor of ExecuteNetwork. Now input name, input type, output name, output type and model type are read from the model.

    Arm NN Build Tool:

    • Introduced Arm NN Build Tool which consists of an official Arm NN Dockerfile for building Arm NN and Arm Compute Library (ACL).
    • This tool replaces the majority of our existing build guides as a user-friendly way to build Arm NN (and its dependencies) from scratch.
    • Tested on x86_64 (Intel) and aarch64 (Arm) build hosts for the Ubuntu platform.
    • Currently supports targeting Linux devices (from Ubuntu 18.04 onwards) on x86_64, aarch32 and aarch64 architectures.

    Bug Fixes

    • The models in format .armnn (serialized models) were failing in 22.05, this problem has been solved by adding the constant layers before the operator layers.
    • Neon fold padding into average pool 2D quantization bug fix.
    • Fix segmentation fault when running --bf16-turbo-mode on FPGA.

    Other Changes

    • General documentation refactor and updates.
    • Added LICENSE.spdx for Arm NN
    • Delay backend deprecation from 22.11 to 23.08

    ABI/API Changes

    The following front-end API changes have occurred during the implementation of 22.08 that users should be aware of before upgrading.

    .

    Feature SHA Gerrit Review Resultant ABI/API changes
    Import inputs but don't export outputs fails 626bd90 https://review.mlplatform.org/c/ml/armnn/+/7661 Field m_ExportEnabled has been added to type OptimizerOptions. This field will not be initialized by old clients that have not been recompiled.
    Get non-const IConnectableLayer from I/O slots 09fa24d https://review.mlplatform.org/c/ml/armnn/+/7835 Pure virtual method GetOwningIConnectableLayer ( ) has been added to classes IOutputSlot and IInputSlot.
  • Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method.
  • The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    • Remove deprecated code 22.05 4d2eec0 https://review.mlplatform.org/c/ml/armnn/+/7712 Removed Symbols:
  • IsCapabilitySupported ( BackendId const& backend, enum BackendCapability capability ) FullyConnectedDescriptor::GetNumViews ( ) const INetwork::Accept ( ILayerVisitor& visitor ) const
  • Pure virtual method Accept ( ILayerVisitor& ) const has been removed from class IConnectableLayer.
  • The layout of v-table has been changed. Call of this virtual method or any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    		•
    Modified SubgraphView returned by GetWorkingCopy() cea3d49 https://review.mlplatform.org/c/ml/armnn/+/7852 Pure virtual method GetSlotIndex ( ) const has been added to class IInputSlot.
  • Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method.
  • The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    		•
    Update the async api to use ExecutionData 21a6a1a https://review.mlplatform.org/c/ml/armnn/+/7878 experimental::IWorkingMemHandle Pure virtual method GetExecutionDataAt ( unsigned int ) has been added to this class.
  • Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method.
  • The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
  • Pure virtual method GetWorkingMemDescriptor ( LayerGuid ) has been removed from this class.
  • The layout of v-table has been changed. Call of this virtual method or any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.

    • The following back-end API changes have occurred during the implementation of 22.08 that users should be aware of before upgrading.

    Feature SHA Gerrit Review Resultant ABI/API changes
    Update the async api to use ExecutionData 21a6a1a https://review.mlplatform.org/c/ml/armnn/+/8051/2 The following virtual functions have been added to class IBackendInternal:
  • virtual ExecutionData CreateExecutionData(WorkingMemDescriptor&) const
  • virtual void UpdateExecutionData(ExecutionData&, WorkingMemDescriptor&) const
  • The layout of v-table has been changed. Call of this virtual method or any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
  • The signature of IWorkload::ExecuteAsync() has changed, it now accepts ExecutionData& instead of WorkingMemDescriptor&.
    • Add GetMemoryRequirements to IWorkload 5e09080 https://review.mlplatform.org/c/ml/armnn/+/7886 The following virtual function has been added to class IWorkload:
  • virtual armnn::Optionalarmnn::MemoryRequirements GetMemoryRequirements()
  • The layout of v-table has been changed. Call of this virtual method or any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    • Modified SubgraphView returned by GetWorkingCopy() cea3d49 https://review.mlplatform.org/c/ml/armnn/+/7852 The signature of SubgraphView::GetWorkingCopy() has changed, it has now been marked as const to reflect the fact that the graph represented by the working copy does not get altered.

    TfLite Delegate

    New features

    • Added support for LOG
    • Added support for SIN
    • Add JNI interface

    Bug Fixes

    • Fix running MobileBERT on CpuRef
    • Only use the macro ARMNN_TFLITE_DELEGATE
    • DelegateQuickStartGuide.md errors fix

    PyArmNN

    • Documentation update running PyArm NN with ONNX parser.

    Build Dependencies

    Tools Supported Version
    Git 2.17.1 or later
    SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
    Cmake 3.19.0
    Tensorflow 2.5.0
    Onnx 1.6.0
    Flatbuffer 1.12.0
    Protobuf 3.12.0
    Android NDK r20b
    mapbox/variant 1.2.0
    cxxopts SHA 12e496da3d486b87fa9df43edea65232ed852510
    doctest 2.4.6
    fmt 7.0.1
    ghc 1.3.2
    half 1.12.0
    stb 2.16
    Assets 11

    Release 22.05.01

    20 Jun 08:17
    @nikraj01 nikraj01
    v22.05.01
    4a1a475
    Compare
    Choose a tag to compare
    Release 22.05.01

    Summary

    New Features

    This is a patch release of 22.05 where we have implemented Pooling3d custom operator for ArmNN TfLite Delegate. This feature is available in the 22.05 release branch itself (branches/armnn_22_05) and in the tag created for patch release v22.05.01.

    Assets 14

    Release 22.05

    26 May 10:32
    @nikraj01 nikraj01
    v22.05
    7bbb79b
    Compare
    Choose a tag to compare
    Release 22.05

    Summary

    New Features

    • ArmnnTestUtils is now versioned and under ABI compliance checker
    • Added support for Int32 CONCATENATION layer for CpuRef
    • Added support for Float32 Unidirectional Sequence LSTM layer for CpuAcc and GpuAcc
    • Added support for GatherNd for CpuRef, CpuAcc and GpuAcc
    • Added support for SQRT for CpuAcc and GpuAcc
    • Added support for Depthwise Convolution2d ConstTensorsAsInput for CpuRef, CpuAcc and GpuAcc
    • Added support for Conv2d ConstTensorsAsInput for CpuRef, CpuAcc and GpuAcc
    • Added support for Fully Connected ConstTensorsAsInput for CpuAcc and GpuAcc
    • Added support for MaxPool3D and AveragePool3D for CpuAcc and GpuAcc
    • Added support for L2Pooling3D for GpuAcc
    • Added support for UnidirectionalLSTM for CpuAcc
    • ConstTensorsAsInput: Optimizer Fix - FuseBatchNorm
    • ConstTensorsAsInput: Optimizer Fix - FoldPadIntoConvolution2d
    • ConstTensorsAsInput: Optimizer Fix - Fp32ToBf16 optimization

    TfLite Parser

    • Added support for GatherNd
    • Added support for FloorDiv
    • Added support for UnidirectionalLSTM
    • Do not create Floor for FloorDiv layer when the data type is int32

    ArmNN Serializer/Deserializer

    • Added support for GatherNd

    ExecuteNetwork App Changes:

    • Added Reuse IO Buffers mode
    • Profiling details weights and bias JSON keys deprecated. Will be removed for 22.08

    Bug Fixes

    • Fixed crashing in profiling
    • Fixed the issue with running SimpleSample app in Raspi
    • Removed MockBackend.hpp from armnn/src/backends/backendsCommon/test/ to solve problems when using Visual Studio in Windows
    • Fixed segfault in RefDepthwiseConvolution2d workload

    Other Changes

    • ArmNN Baremetal
      • Change the namespace from armnn::profiling to arm::pipe

    ABI/API Changes

    The following front-end API changes have occurred during the implementation of 22.05 that users should be aware of before upgrading.

    .

    Feature SHA Gerrit Review Resultant ABI/API changes
    Change the namespace from armnn::profiling to arm::pipe 5aa9fd7 https://review.mlplatform.org/c/ml/armnn/+/7222
  • Pure virtual method GetOwningIConnectableLayer( ) const has been added to class IOutputSlot. Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method.
  • The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
  • The following functions has had a change in signature meaning it will not be recognized by old applications: BackendRegistry::SetProfilingService
    IRuntime::RegisterDebugCallback
  • Type of field m_LocalPacketHandlers has been changed from std::vector<std::shared_ptrprofiling::ILocalPacketHandler > to std::vector<std::shared_ptrarm::pipe::ILocalPacketHandler > in Runtime::CreateOptions::ExternalProfilingOptions
  • Type of return value has been changed from profiling::ProfilingGuid to arm::pipe::ProfilingGuid in OptimizedNetwork::GetGuid
    • Replace ProfilingService includes with IProfilingService. af94772 https://review.mlplatform.org/c/ml/armnn/+/7240 The following function has had a change in signature meaning it will not be recognized by old applications.
    BackendRegistry::SetProfilingService
    Remove dependency on armnn::Exception classes from the Profiling code f9db3ef https://review.mlplatform.org/c/ml/armnn/+/7280 Class armnn::BackendProfilingException has been moved to namespace arm::pipe; this will result in older applications not being able to find it.
    Replace armnn:Optional with arm::pipe::Optional in profiling code decd08b https://review.mlplatform.org/c/ml/armnn/+/7295 Class armnn::TimeoutException has been moved to namespace arm::pipe; this will result in older applications not being able to find it.
    Add Unidirectional Sequence Lstm support to TFLite 5880b91 https://review.mlplatform.org/c/ml/armnn/+/7023 Following fields have been added to struct LstmDescriptor:
    m_CellIntermediateScale
    m_ForgetIntermediateScale
    m_HiddenStateScale
    m_HiddenStateZeroPoint
    m_InputIntermediateScale
    m_OutputIntermediateScale
    As a result of this size of the struct has been changed
    ConstTensorsAsInput: DepthwiseConvolution2d 0690265 https://review.mlplatform.org/c/ml/armnn/+/7417 Pure virtual method VisitDepthwiseConvolution2dLayer ( IConnectableLayer const*, struct DepthwiseConvolution2dDescriptor const&, char const* ) has been added to this class..
  • Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method..
  • The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    		•
    ConstTensorsAsInput: Conv2d - FrontEnd b4dd5cc https://review.mlplatform.org/c/ml/armnn/+/7382 Pure virtual method VisitConvolution2dLayer ( IConnectableLayer const*, struct Convolution2dDescriptor const&, char const* ) has been added to this class.
  • Applications will not provide the implementation for this pure virtual method and therefore cause a crash in the library trying to call this method.
  • The layout of v-table has been changed. Call of any virtual method at higher position in this class or its subclasses may result in crash or incorrect behavior of applications.
    		•

    The following back-end API changes have occurred during the implementation of 22.02 that users should be aware of before upgrading.

    Feature SHA Gerrit Review Resultant ABI/API changes
    Move headers to profiling/client/include 2776183 https://review.mlplatform.org/c/ml/armnn/+/7327 Headers have been moved to profiling/client/include.
    Change the namespace from armnn::profiling to arm::pipe 5aa9fd7 https://review.mlplatform.org/c/ml/armnn/+/7222
  • Namespace changed from armnn: profiling to armnn: pipe:: profiling
    		•

    TfLite Delegate

    New features

    • Added support for GatherNd

    Bug Fixes

    Note: Arm NN is aware of an issue where converting a model to .armnn will yield unpredictable results when reading back in through the deserializer. This is due to the serializer being dependent on graph topology and the graph being out of order. The graph becomes out of order because of the additional constant layers as inputs that are created through the parsers

    PyArmNN

    • Added support for GatherNd
    • Added Pooling3D

    Build Dependencies

    Tools Supported Version
    Git 2.17.1 or later
    SCons 2.4.1 (Ubuntu) 2.5.1 (Debian)
    Cmake 3.5.1 (Ubuntu) and 3.7.2 (Debian)
    Tensorflow 2.5.0
    Onnx 1.6.0
    Flatbuffer 1.12.0
    Protobuf 3.12.0
    Android NDK r20b
    mapbox/variant 1.2.0
    cxxopts SHA 12e496da3d486b87fa9df43edea65232ed852510
    doctest 2.4.6
    fmt 7.0.1
    ghc 1.3.2
    half 1.12.0
    stb 2.16

    Android 12 Compatibility Testing was performed using the following:

    Android Tag Android Build ID Mali Driver Android Compatibility Test Suite Android Vendor Test Suite
    android-12.0.0_r1 SP1A.210812.015 r36p0_01eac0-rc0 12_r2 (7987736) 12_r2 (7973604)

    Android 11 Compatibility Testing was performed using the following:

    Android Tag Android Build ID Mali Driver Android Compatibility Test Suite Android Vendor Test Suite
    android-11.0.0_r6 RPM1.210413.002 r33p0_01eac0 11_r5 (7640833) 11_r5 (7599184)

    Android 10 Compatibility Testing was performed using the following:

    Androidtag Android Build ID Mali Driver
    android-10.0.0_r39 QQ3A.200605.002.A1 R23P0_01REL0
    Assets 2