Modify rule S6929: add the PyTorch library (#3984)

This commit is contained in:
Ghislain Piot 2024-09-27 10:51:21 +00:00 committed by GitHub
parent f9b1835f76
commit b9b85c7a80
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 44 additions and 5 deletions

View File

@ -123,6 +123,7 @@
* SignXML * SignXML
* SQLAlchemy * SQLAlchemy
* ssl * ssl
* TensorFlow
// Docker // Docker
* Wget * Wget
// Cloudformation // Cloudformation

View File

@ -1,5 +1,5 @@
{ {
"title": "The axis argument should be specified when using TensorFlow's reduction operations", "title": "The reduction axis/dimension should be specified when using reduction operations",
"type": "CODE_SMELL", "type": "CODE_SMELL",
"status": "ready", "status": "ready",
"remediation": { "remediation": {
@ -7,13 +7,17 @@
"constantCost": "5min" "constantCost": "5min"
}, },
"tags": [ "tags": [
"tensorflow",
"pytorch",
"machine-learning",
"scientific-computing"
], ],
"defaultSeverity": "Major", "defaultSeverity": "Major",
"ruleSpecification": "RSPEC-6929", "ruleSpecification": "RSPEC-6929",
"sqKey": "S6929", "sqKey": "S6929",
"scope": "All", "scope": "All",
"defaultQualityProfiles": ["Sonar way"], "defaultQualityProfiles": ["Sonar way"],
"quickfix": "unknown", "quickfix": "targeted",
"code": { "code": {
"impacts": { "impacts": {
"MAINTAINABILITY": "MEDIUM", "MAINTAINABILITY": "MEDIUM",

View File

@ -1,8 +1,10 @@
This rule raises an issue when the axis argument is not provided to TensorFlow's reduction operations. This rule raises an issue when the `axis`/`dim`` argument is not provided to reduction operations.
== Why is this an issue? == Why is this an issue?
The result of TensorFlow's reduction operations (i.e. ``tf.math.reduce_sum``, ``tf.math.reduce_std``), === TensorFlow
The result of reduction operations (i.e. ``tf.math.reduce_sum``, ``tf.math.reduce_std``, ``torch.sum``, ``torch.mean``, etc...),
highly depends on the shape of the Tensor provided. highly depends on the shape of the Tensor provided.
[source,python] [source,python]
@ -61,7 +63,11 @@ In the example above, specifying the axis clarifies the intent, as the result no
reduce across all dimensions the user should provide the list of axis `axis=[0,1]` reduce across all dimensions the user should provide the list of axis `axis=[0,1]`
or clearly state the default behavior should be applied with ``axis=None``. or clearly state the default behavior should be applied with ``axis=None``.
== How to fix it === The PyTorch equivalent
The same behavior occurs in PyTorch, but the argument is called `dim` instead of `axis`.
== How to fix it in TensorFlow
To fix this issue provide the axis argument when using a TensorFlow reduction operation such as ``tf.math.reduce_sum``, ``tf.math.reduce_prod``, ``tf.math.reduce_mean``, etc... To fix this issue provide the axis argument when using a TensorFlow reduction operation such as ``tf.math.reduce_sum``, ``tf.math.reduce_prod``, ``tf.math.reduce_mean``, etc...
@ -88,6 +94,32 @@ tf.math.reduce_sum(x, axis=0) # Compliant: the reduction will happen only on the
---- ----
== How to fix it in PyTorch
To fix this issue provide the dim argument when using a PyTorch reduction operation such as ``torch.sum``, ``torch.prod``, ``torch.mean``, etc...
=== Code examples
==== Noncompliant code example
[source,python,diff-id=2,diff-type=noncompliant]
----
import torch
x = torch.tensor([[1, 1, 1], [1, 1, 1]])
torch.sum(x) # Noncompliant: the dim argument defaults to None
----
==== Compliant solution
[source,python,diff-id=2,diff-type=compliant]
----
import torch
x = torch.tensor([[1, 1, 1], [1, 1, 1]])
torch.sum(x, dim=None) # Compliant: all dimensions will be reduced
----
== Resources == Resources
=== Documentation === Documentation
@ -99,6 +131,8 @@ tf.math.reduce_sum(x, axis=0) # Compliant: the reduction will happen only on the
* TensorFlow Documentation - https://www.tensorflow.org/api_docs/python/tf/math/reduce_sum[tf.math.reduce_sum reference] * TensorFlow Documentation - https://www.tensorflow.org/api_docs/python/tf/math/reduce_sum[tf.math.reduce_sum reference]
* TensorFlow Documentation - https://www.tensorflow.org/api_docs/python/tf/math/reduce_variance[tf.math.reduce_variance reference] * TensorFlow Documentation - https://www.tensorflow.org/api_docs/python/tf/math/reduce_variance[tf.math.reduce_variance reference]
* PyTorch Documentation - https://pytorch.org/docs/stable/torch.html#reduction-ops[Reduction operations]
=== Articles & blog posts === Articles & blog posts
* Vahidk Developers Guide - https://github.com/vahidk/EffectiveTensorflow?tab=readme-ov-file#broadcasting-the-good-and-the-ugly[Broadcasting the good and the ugly] * Vahidk Developers Guide - https://github.com/vahidk/EffectiveTensorflow?tab=readme-ov-file#broadcasting-the-good-and-the-ugly[Broadcasting the good and the ugly]