Linear Models#

In this first chapter of semi-parametric models, we introduce the linear non-Gaussian acyclic model (LiNGAM) [SHHyvarinen+06]. We will show that we are able to retrieve the full causal structure of the data-generating process, if we are willing to make some relatively mild assumptions about the functional form of the structural causal model or the distribution of the errors.

The LiNGAM Model#

The LiNGAM model for \(p\) observed variables \(x_1, x_2, \ldots, x_p\) is given by

(6)#\[\begin{equation} x_i = \sum_{j \in \operatorname{pa}(x_i)}b_{ij}x_j + e_i \quad (i=1, \ldots, p) \end{equation}\]

where each observed variable \(x_i\) is a linear sum of their parent variables \(\operatorname{pa}(x_i)\) plus some noise \(e_i\). If the coefficient \(b_{ij}\) is zero, then there is no direct causal effect from \(x_j\) to \(x_i\). The error variables \(e_i\) are independent and follow continuous non-Gaussian distributions. The independence means there are no unobserved or hidden common causes.

In matrix notation, the model is given by

(7)#\[\begin{equation} \mathbf{x} = \mathbf{B}\mathbf{x} + \mathbf{e} \end{equation}\]

where \(\mathbf{x}\) and \(\mathbf{e}\) are \(p\)-dimensional vectors, and \(\mathbf{B}\) is a \(p\times p\) matrix that contains the \(b_{ij}\) coefficients, with \(i, j=1, \ldots, p\).

Let us consider a simple example with three variables. To ease the introduction of the LiNGAM model, we will use the variables from the chain example of the previous chapters, where:

  • Wind forecast errors are directly influences by sudden changes in the weather.

  • Balancing costs are influenced by wind forecast errors. For example, if the system is short because it was expecting a larger production from wind farms, it might incur balancing costs to procure electricity from alternative sources.

  • Weather fluctuarions are influenced by external factors and serve as an exogenous variable in this model.

The true DAG of the data-generating process is given by:

import numpy as np
from lingam.utils import make_dot

# Matrix of coefficients (weights)
m = np.array([[0.0, 0.0, 0.0],
              [1.5, 0.0, 0.0],
              [0.0, 1.2, 0.0]])

# Plotting causal graph
make_dot(m, labels=["weather \nfluctuations", "wind forecast \nerrors", "balancing \ncosts"])
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
../_images/516eab5ea2ac5b1328d958b2e37254521bb18ea3e8900f63d69d03fd26d996b4.svg

From this DAG, the SCM can be written as

(8)#\[\begin{align} \text{wind forecast errors} &= 1.5 \times \text{weather flucturations} + e_f \\ \text{balancing costs} &= 1.2 \times \text{wind forecast errors} + e_b \\ \text{weather fluctuations} &= e_w \\ \end{align}\]

Then, in matrix notation, this model is given by

(9)#\[\begin{equation} \left[\begin{array}{c} \text{wind forecast errors} \\ \text{balancing costs} \\ \text{weather fluctuations} \end{array}\right] = \left[\begin{array}{ccc} 0 & 0 & 1.5 \\ 1.2 & 0 & 0 \\ 0 & 0 & 0 \end{array}\right] \left[\begin{array}{c} \text{wind forecast errors} \\ \text{balancing costs} \\ \text{weather fluctuations} \end{array}\right] + \left[\begin{array}{c} e_f \\ e_b \\ e_w \end{array}\right] \end{equation}\]

Typically, we order the the structural equations according to the true causal order, so that the matrix \(\mathbf{B}\) is permuted to be a lower triangular matrix (this property is used later on to identify the true structure), with all the diagonal elements equal to zero (strictly lower triangular). This simply means that we rewrite the structural equations as

(10)#\[\begin{align} \text{weather fluctuations} &= e_w \\ \text{wind forecast errors} &= 1.5 \times \text{weather flucturations} + e_f \\ \text{balancing costs} &= 1.2 \times \text{wind forecast errors} + e_b \\ \end{align}\]

that, in matrix notation, becomes

(11)#\[\begin{equation} \left[\begin{array}{c} \text{weather fluctuations} \\ \text{wind forecast errors} \\ \text{balancing costs} \end{array}\right] = \left[\begin{array}{ccc} 0 & 0 & 0 \\ 1.5 & 0 & 0 \\ 0 & 1.2 & 0 \end{array}\right] \left[\begin{array}{c} \text{weather fluctuations} \\ \text{wind forecast errors} \\ \text{balancing costs} \end{array}\right] + \left[\begin{array}{c} e_w \\ e_f \\ e_b \end{array}\right] \end{equation}\]

Now, we can rewrite the LiNGAM model as

(12)#\[\begin{equation} x_i = \sum_{j:k(j)<k(i)}b_{ij}x_j + e_i \quad (i=1, \ldots, p) \end{equation}\]

where \(k(\cdot)\) represents the causal ordering, meaning that each variable \(x_i\) is a linear sum of the \(x_j\) variables observed earlier in the causal graph (\(k(j)<k(i)\)), plus its own error variable \(e_i\). We can now plot the causal graph, using the LiNGAM library.

Direct Estimation Method#

Now that we explained the basic formulation, we explain how to estimate the model from the data. There are two main approaches to estimate a LiNGAM model: a direct estimation method (DirectLiNGAM) through a series of regressions and independence tests, and a method based on independent component analysis (ICA) (ICA-based LinGAM). In the case of linear models, the key assumption is that the errors are non-Gaussians. This is the most important assumption as it allows us to identify the causal structure, both using DirectLiNGAM and ICA-based LiNGAM.

Now, we will explain the intuition behing DirectLiNGAM, an iterative procedure that employs a series of regressions and independence tests to directly identify the causal order among the observed variables. It leverages the non-Gaussianity of the data and the assumption of linear causality to systematically test for independence between variables and their residuals from regressions.

The key intuition is that, with non-Gaussian errors, the residuals in the anti-causal direction will not be independent of the predictor. We will see how this works with a practical example.

Exploring the Residuals with Gaussian errors#

Let us consider a simple example, like the one we saw above, but for simplicity we only keep two variables:

(13)#\[\begin{align} \text{weather fluctuations} &= e_w \\ \text{wind forecast errors} &= 1.5 \times \text{weather flucturations} + e_f \\ \end{align}\]

We now assume Gaussian errors, so we have that \(e_w, e_f \sim \mathcal{N}(0, 1)\).

We will show that, in this case, by analysing the residuals we cannot understand the true causal direction, because we can both predict the wind from the prices and the prices from the wind.

import statsmodels.api as sm
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns

# Synthetic data generation based on the given equations
n = 10000
weather_fluctuations = np.random.normal(0, 1, size=n)  # Predictor variable
forecast_errors = 1.5 * weather_fluctuations + np.random.normal(0, 1, size=n)  # Response variable with non-Gaussian noise (uniform)

# Prepare the data for regression
weather_fluctuations_with_const = sm.add_constant(weather_fluctuations)  # Add a constant term for the intercept
forecast_errors_with_const = sm.add_constant(forecast_errors)  # Add a constant term for the intercept

# Linear regression in the true causal direction (wind -> price)
true_model = sm.OLS(endog=forecast_errors, exog=weather_fluctuations_with_const)
true_model_results = true_model.fit()
forecast_errors_pred = true_model_results.predict(weather_fluctuations_with_const)
forecast_errors_residuals = forecast_errors - forecast_errors_pred

# Linear regression in the anti-causal direction (price -> wind)
anti_model = sm.OLS(endog=weather_fluctuations, exog=forecast_errors_with_const)
anti_model_results = anti_model.fit()
weather_fluctuations_pred = anti_model_results.predict(forecast_errors_with_const)
weather_fluctuations_residuals = weather_fluctuations - weather_fluctuations_pred

# Setting up the plots
fig, axes = plt.subplots(1, 2, figsize=(12, 5))

# All the data
hb1 = axes[0].hexbin(weather_fluctuations, forecast_errors_residuals, gridsize=50, cmap='Blues', bins='log')
axes[0].set_title('Residuals in the true causal direction\n(weather fluctuations -> forecast errors)')
axes[0].set_xlabel('Weather')
axes[0].set_ylabel('Residuals')

# After conditioning
hb2 = axes[1].hexbin(forecast_errors, weather_fluctuations_residuals, gridsize=50, cmap='Blues', bins='log')
axes[1].set_title('Residuals in the anti-causal direction\n(forecast errors -> weather fluctuations)')
axes[1].set_xlabel('Forecast errors')
axes[1].set_ylabel('Residuals')
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Text(0, 0.5, 'Residuals')
../_images/aa063a05602bc9afd13558913f2fffaf82bdd4ec682c1bebe0a53ed56557f05d.png

Where we can see how the residuals are independent of the input variable in both the true and anti-causal directions.

Exploring the Residuals with non-Gaussian errors#

Now, by simply assuming the errors are uniformly distributed, we can wee how it will be possible to detect the true causal direction by testing the independence of the residuals from the input variable.

# Synthetic data generation based on the given equations
n = 10000
weather_fluctuations = np.random.uniform(0, 1, size=n)  # Predictor variable
forecast_errors = 1.5 * weather_fluctuations + np.random.uniform(0, 1, size=n)  # Response variable with non-Gaussian noise (uniform)

# Prepare the data for regression
weather_fluctuations_with_const = sm.add_constant(weather_fluctuations)  # Add a constant term for the intercept
forecast_errors_with_const = sm.add_constant(forecast_errors)  # Add a constant term for the intercept

# Linear regression in the true causal direction (wind -> price)
true_model = sm.OLS(endog=forecast_errors, exog=weather_fluctuations_with_const)
true_model_results = true_model.fit()
forecast_errors_pred = true_model_results.predict(weather_fluctuations_with_const)
forecast_errors_residuals = forecast_errors - forecast_errors_pred

# Linear regression in the anti-causal direction (price -> wind)
anti_model = sm.OLS(endog=weather_fluctuations, exog=forecast_errors_with_const)
anti_model_results = anti_model.fit()
weather_fluctuations_pred = anti_model_results.predict(forecast_errors_with_const)
weather_fluctuations_residuals = weather_fluctuations - weather_fluctuations_pred

# Setting up the plots
fig, axes = plt.subplots(1, 2, figsize=(12, 5))

# All the data
hb1 = axes[0].hexbin(weather_fluctuations, forecast_errors_residuals, gridsize=50, cmap='Blues', bins='log')
axes[0].set_title('Residuals in the true causal direction\n(weather fluctuations -> forecast errors)')
axes[0].set_xlabel('Weather')
axes[0].set_ylabel('Residuals')

# After conditioning
hb2 = axes[1].hexbin(forecast_errors, weather_fluctuations_residuals, gridsize=50, cmap='Blues', bins='log')
axes[1].set_title('Residuals in the anti-causal direction\n(forecast errors -> weather fluctuations)')
axes[1].set_xlabel('Forecast errors')
axes[1].set_ylabel('Residuals')
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Text(0, 0.5, 'Residuals')
../_images/523900598a349f5349aeb66e78d3738424be0cf3557c12b8ba0fa5133c08cde8.png

Example#

Now, we try a full estimation example, where we have a slightly more complex causal structure, that we will need to retrieve from observational data. We now add to the model:

  • Outages: an exogenous variable affecting the balancing costs.

  • heating demand: a variable affected by changes in the weather, which affects the balancing costs.

Assume the following structural equations:

(14)#\[\begin{align} \text{weather fluctuations} &= e_w \\ \text{outages} &= e_o \\ \text{heating demand} &= 0.3 \times \text{weather flucturations} + e_h \\ \text{wind forecast errors} &= 1.5 \times \text{weather flucturations} + e_f \\ \text{balancing costs} &= 1.5 \times \text{outages} + 0.7 \times \text{heating demand} + 1.2 \times \text{wind forecast errors} + e_b \\ \end{align}\]

where the errors are now assumed to be uniformly distributed.

Then, the true causal graph is given by

# Matrix of coefficients (weights)
B = np.array([
    [0.0, 0.0, 0.0, 0.0, 0.0],   # Weather fluctuations
    [0.0, 0.0, 0.0, 0.0, 0.0],   # Outages
    [0.3, 0.0, 0.0, 0.0, 0.0],   # Heating demand
    [1.5, 0.0, 0.0, 0.0, 0.0],   # Wind forecast errors
    [0.0, 1.5, 0.7, 1.2, 0.0]    # Balancing costs
])

# Plotting causal graph
labels = ["weather \nfluctuations", "outages", "heating \ndemand", "wind forecast \nerrors", "balancing \ncosts"]
make_dot(B, labels=labels)
../_images/09dce491da59bfb8e5ac3eb7f78d7b6bead63bd28d1e8f2ddf843c5810dfacb8.svg

Let us now generate some data from this causal graph.

import pandas as pd

# Set the random seed for reproducibility
np.random.seed(42)

# Number of samples
n = 1000

# Generating synthetic data based on the specified equations with uniform errors
weather_fluctuations = np.random.uniform(0, 1, size=n) 
outages = np.random.uniform(0, 1, size=n)
forecast_errors = 1.5 * weather_fluctuations + np.random.uniform(0, 1, size=n)
heating = 0.3 * weather_fluctuations + np.random.uniform(0, 1, size=n)
balancing_costs = 0.7 * heating + 1.2 * forecast_errors + 1.5 * outages + np.random.uniform(0, 1, size=n)

# Creating a DataFrame to hold the generated data
data = pd.DataFrame({'weather fluctuations': weather_fluctuations, 'outages': outages, 'wind forecast errors': forecast_errors,
                     'heating demand': heating, 'balancing costs': balancing_costs})

Now, we fit the DirectLiNGAM model and plot the estimated causal structure. We will use the implementation offered in the original LiNGAM package [IIZ+23].

import lingam

direct_model = lingam.DirectLiNGAM()
direct_model.fit(data)
make_dot(direct_model.adjacency_matrix_, labels=labels)
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
../_images/2841a14c06717bb8c9daf3f21720453269cbc83feb5df8cceeb55e7ff2c2b9d4.svg

We can see how we successfully recovered the true casual structure, even though the model did not find the exact coefficients for the wind forecast errors and heating demand.

ICA Estimation Method#

Introduction to ICA#

ICA is a computational method, developed in the field of signal processing, to separate a multivariate signal into additive subcomponents assuming that these subcomponents are non-Gaussian signals and statistically independent from each other [HyvarinenO00]. Consider a simple ICA model given by

(15)#\[\begin{align} x_1 &= a_{11}s_1 + a_{12}s_2 \\ x_2 &= a_{21}s_1 + a_{22}s_2 \end{align}\]

where \(x_1\) and \(x_2\) are the observed variables, and \(s_1\) and \(s_2\) are continuous hidden variables. Assuming that \(s_1\) and \(s_2\) are independent and non-Gaussian, ICA attempts to recover them from the observed matrix \(\mathbf{X}\). In matrix form, we can write it as

(16)#\[\begin{equation} \mathbf{x} = \mathbf{A}\mathbf{s} \end{equation}\]

In our example, it would mean

(17)#\[\begin{equation} \left[\begin{array}{c} x_1 \\ x_2 \end{array}\right] =\left[\begin{array}{cc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array}\right] \left[\begin{array}{c} s_1 \\ s_2 \end{array}\right] \end{equation}\]

The \(p \times q\) mixing matrix \(\mathbf{A}\) represents how the unobserved components \(\mathbf{s}\) are mixed to generate the observed variables \(\mathbf{x}\). Any two columns of \(\mathbf{A}\) are assumed to be linearly independent. The mixing matrix \(\mathbf{A}\) is identifiable, except for the ordering and scaling of its columns.

Let’s show how ICA works with a practical example. We start by generating the original signals \(s_1\) and \(s_2\)

from sklearn.decomposition import FastICA
import numpy as np
import matplotlib.pyplot as plt

np.random.seed(0)
sample_size = 1000
time = np.linspace(0, 24, sample_size)
s1 = np.random.uniform(low=0, high=1, size=sample_size)
s2 = np.sign(np.sin(3 * time))  # Signal 2: square signal
S = np.c_[s1, s2]
plt.figure(figsize=(12, 4))
plt.plot(time, S)
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.show()
../_images/4854eea8af2d66092a8270f8976873572568e7fa0da31ac46870f360857e7712.png

We now mix the signals, to generate the observed matrix \(\mathbf{X}\)

# Mix data
A = np.array([[1, 1], [0.5, 2]])  # Mixing matrix
X = np.dot(S, A.T)  # Generate observations
plt.figure(figsize=(12, 4))
plt.plot(time, X)
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.show()
../_images/569a2329ff25d95059be5325cb4c7bb68d4c5958d18b40146c1225276bbeeae9.png

Now, we apply ICA to try and retrieve the original signals.

ica = FastICA(n_components=2)
S_ = ica.fit_transform(X)  # Reconstruct signals
A_ = ica.mixing_  # Get estimated mixing matrix
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
plt.figure(figsize=(12, 4))
plt.plot(time, S_)
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.show()
../_images/8cb2d56b1c88c9622b9668ea1dc462f71b6d11248db412f0b98a5269a02cd147.png

As you can see, with ICA we are able to retrieve the original signals up to their order and scale because we assume the sources are non-Gaussian and statistically independent. The order of the components might differ from the original, and the scale might also differ due to the nature of the ICA algorithm.

ICA-based LiNGAM#

Going back to the LiNGAM model, we had:

(18)#\[\begin{equation} \mathbf{x} = \mathbf{B}\mathbf{x} + \mathbf{e} \end{equation}\]

where \(\mathbf{x}\) and \(\mathbf{e}\) are \(p\)-dimensional vectors, and \(\mathbf{B}\) is a \(p\times p\) matrix that contains the \(b_{ij}\) coefficients, with \(i, j=1, \ldots, p\). We can solve the equation \(\mathbf{x} = \mathbf{B}\mathbf{x} + \mathbf{e}\) for the vector of observed variables \(\mathbf{x}\) as in

(19)#\[\begin{equation} (\mathbf{I}- \mathbf{B})\mathbf{x} = \mathbf{e} \end{equation}\]

Then, if we left-multiply both sides by the inverse of \(\mathbf{I}- \mathbf{B}\), we get

(20)#\[\begin{align} (\mathbf{I}- \mathbf{B})^{-1}(\mathbf{I}- \mathbf{B})\mathbf{x} &= (\mathbf{I}- \mathbf{B})^{-1}\mathbf{e} \\ \mathbf{x} &= (\mathbf{I}- \mathbf{B})^{-1}\mathbf{e} \\ \mathbf{x} &= \mathbf{A}\mathbf{e} \end{align}\]

Now, since we assumed the errors are independent and non-Gaussian, we have that this equation corresponds to the ICA model, and the matrix \(\mathbf{A} = (\mathbf{I}- \mathbf{B})^{-1}\) corresponds to the mixing matrix in ICA. This gives the hint of why we can use ICA to estimate the LiNGAM model. The cool part is that, in this case, using the acyclicity assumption, we can also find the coefficients in terms of the right ordering and scaling.

Example#

Now, we can see that using ICA we can estimate the causal model, in the same way we did for the Direct estimation method, using the LiNGAM package [IIZ+23].

# ICA-based LiNGAM
ica_model = lingam.ICALiNGAM()
ica_model.fit(data)
make_dot(ica_model.adjacency_matrix_, labels=labels)
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
../_images/2841a14c06717bb8c9daf3f21720453269cbc83feb5df8cceeb55e7ff2c2b9d4.svg