---
type: archive
area: archive
status: archived
date: 2026-05-03
created: 2026-05-03
updated: 1980-01-01
tags:
  - archive
---
## What Entropy Means in Data Analysis

In data analysis, **entropy** is a concept borrowed from information theory and statistics that is used to measure the amount of uncertainty or disorder in a dataset or a probability distribution. It provides a way to quantify the unpredictability or randomness in the data. The concept of entropy is widely used in various fields of data analysis, including machine learning, information retrieval, and statistics. Here's what entropy means in data analysis:

1. **Information Gain**: In machine learning, entropy is used in decision tree algorithms to quantify the amount of information gained by splitting a dataset into subsets based on a feature. It helps in selecting the most informative features for classification or regression tasks.

2. **Feature Selection**: Entropy can be used to rank or select features in feature selection processes, where you want to identify the most relevant features for a given task by considering the entropy of the target variable with respect to different features.

3. **Clustering**: In clustering, entropy can be used as a measure of cluster purity. Lower entropy values indicate that the clusters are more homogeneous, while higher entropy values suggest that the data points are more mixed.

4. **Information Retrieval**: In information retrieval systems, entropy is used to rank the relevance of documents or search results based on the distribution of relevant and non-relevant items.

5. **Probability Distributions**: Entropy is used to characterize the spread or randomness of a probability distribution. In this context, it measures the uncertainty associated with the occurrence of different events.

The entropy of a dataset or a probability distribution is calculated using mathematical formulas, such as the Shannon entropy formula:

H(X) = -Σ [p(x) * log2(p(x))]

### Sturge

![[img-20231221-141550-64ba20fa.png]]

### Scott
![[img-20231221-164802-78a4b859.png]]

df.std() // ecartType

nbr_bins = df.max() - df.min())/(3.5 * df.std() * pow(len(df),-1/3)

#### nuages de points
scatter plot

![[Pasted image 20231221153705.png]]