Introduction to Testing DAG Validity: Local Markov and Edge Dependence Tests

3 June, 2024

In the realm of data science and causal inference, Directed Acyclic Graphs (DAGs) are powerful tools for modeling the causal relationships between variables. However, creating a DAG is only the first step. To ensure the accuracy and reliability of the causal inferences drawn from these models, we need to validate that the DAG accurately represents the underlying data-generating process. This is where the Local Markov Test and the Edge Dependence Test come into play.

Imagine you’re working on a project to understand how various factors influence customer satisfaction in an e-commerce setting. You have a hypothesis about how these factors are causally related, which you’ve represented as a DAG. But how can you be confident that this DAG is a faithful representation of the true causal relationships? How can you verify that the dependencies and independencies implied by your DAG hold in the real world?

The Local Markov Test and the Edge Dependence Test offer robust methods for validating your DAG. The Local Markov Test checks whether each node in your DAG is conditionally independent of its non-descendants given its parents, ensuring that the local structures of your DAG adhere to the expected independencies. On the other hand, the Edge Dependence Test evaluates the strength of the correlations between connected nodes, confirming that the edges in your DAG represent significant dependencies.

In this blog post, we’ll delve into the details of these two essential tests. We’ll explain the intuition behind each test, demonstrate how they are applied using Python, and illustrate their importance with visual examples. By the end of this post, you’ll have a clear understanding of how to use these tests to validate your DAGs and ensure that your causal inferences are built on a solid foundation. Whether you’re a data scientist, a researcher, or anyone interested in causal modeling, mastering these tests will enhance the rigor and reliability of your analyses.

1. Local Markov Test

Intuition

The Local Markov Test checks whether a variable is independent of its non-descendants given its parents in the DAG. This is based on the local Markov property, which states that in a DAG, each node is conditionally independent of its non-descendants given its parents.

Example and Visual Illustration, Consider the following simplified DAG:


    Z1      Z2
     \     /
      \   /
       \ /
        X2
        |
        Y
Here, X2 has parents Z1 and Z2. According to the local Markov property, X2 should be independent of its non-descendants (in this case, Y) given its parents (Z1 and Z2).

Steps in the Local Markov Test:

Identify the variable (X2) and its parents (Z1 and Z2).
Condition on the parents (Z1 and Z2) and check if X2 is independent of its non-descendants (Y).
Perform a statistical test (e.g., kernel-based test) to see if the conditional independence holds.
Parents of X2: Z1, Z2
Non-descendant of X2: Y
Test: X2 ⊥ Y | {Z1, Z2}
If X2 is conditionally independent of Y given Z1 and Z2, it supports the DAG structure.

If the p-value of the test is high, it suggests that X2 is indeed independent of Y given Z1 and Z2, supporting the DAG structure.

2. Edge Dependence Test

Intuition

The Edge Dependence Test checks if the edges in the DAG carry a significant amount of correlation, confirming the dependencies between connected nodes. This test verifies that the presence of an edge (or the lack of it) corresponds to the observed dependencies in the data.

Example and Visual Illustration, Consider the same DAG:

Here, we have edges Z1 -> X2, Z2 -> X2, and X2 -> Y. The edge dependence test will check if these edges reflect actual dependencies.

Steps in the Edge Dependence Test:

Identify the edges in the DAG.
Check the correlation between the connected nodes.
Perform a statistical test to see if the correlation is significant.

If the p-values are low, it suggests that there is a significant correlation between connected nodes, supporting the DAG structure.

Example Outputs Explained

import networkx as nx
import dowhy.gcm as gcm

digraph = nx.DiGraph([('Z1','X1'),
                    ('X1','D'),
                    ('Z1','X2'),
                    ('Z2','X3'),
                    ('X3','Y'),
                    ('Z2','X2'),
                    ('X2', 'Y'),
                    ('X2', 'D'),
                    ('M', 'Y'),
                    ('D', 'M')])

from dowhy.gcm.independence_test import kernel_based, regression_based

causal_model = gcm.StructuralCausalModel(digraph)
rej = gcm.refute_causal_structure(causal_model.graph, data, conditional_independence_test=kernel_based)

Local Markov Test Output

'X1': {'local_markov_test': {'p_value': 0.4571576013640475,
    'fdr_adjusted_p_value': 0.4876347747883173,
    'success': True}}

p_value: Probability of observing the data if X1 were independent of its non-descendants given its parents.
fdr_adjusted_p_value: Adjusted p-value for multiple testing.
success: Indicates if the test passed (high p-value suggests independence).

Edge Dependence Test Output

'Y': {'edge_dependence_test': {'M': {'p_value': 0.0,
     'fdr_adjusted_p_value': 0.0,
     'success': True},
    'X2': {'p_value': 0.0, 'fdr_adjusted_p_value': 0.0, 'success': True},
    'X3': {'p_value': 0.0, 'fdr_adjusted_p_value': 0.0, 'success': True}}}

p_value: Probability of observing the data if there were no correlation between the connected nodes.
fdr_adjusted_p_value: Adjusted p-value for multiple testing.
success: Indicates if the test passed (low p-value suggests significant correlation).

Summary

Local Markov Test: Checks if a node is conditionally independent of its non-descendants given its parents.
Edge Dependence Test: Checks if there is a significant correlation between connected nodes.

Both tests help validate the structure of a DAG by ensuring that the implied conditional independencies and dependencies match the observed data. If the tests are passed, it supports the validity of the causal model represented by the DAG. If not, it suggests that the DAG may not accurately represent the causal structure of the data.

Annex: A complete DAG example

Let’s delve into the code with an intuitive explanation, grounded in the concepts of causal inference, directed acyclic graphs (DAGs), and structural equation models (SEMs)

Defining the Causal Graph

The first step is to create a representation of our causal assumptions using a DAG. This involves defining the nodes (variables) and the directed edges (causal relationships) between them.

causal_graph = """digraph {
Z1; Z2; X1; X2; X3; D; M; Y;
Z1 -> X1;
X1 -> D;
Z1 -> X2;
Z2 -> X3;
X3 -> Y;
Z2 -> X2;
X2 -> Y;
X2 -> D;
M -> Y;
D -> M;
}"""
cm = dowhy.CausalModel(data=data, treatment='D', outcome='Y', graph=causal_graph)

Here, we’re encoding our understanding of how different variables influence each other. For instance, Z1 influences X1, D affects M, and M in turn affects Y. This graphical model helps us visualize and reason about these relationships.

Viewing the Model

Visualizing the DAG helps in verifying our assumptions and understanding the flow of causality within the system.

cm.view_model(file_name='dag')

This command generates a visual representation of the DAG, making it easier to see how variables are connected and to check if our model accurately captures our assumptions about causal relationships.

Testing Conditional Independencies

To ensure our DAG is valid, we need to test the conditional independencies implied by the graph. This step involves checking if the relationships (or lack thereof) suggested by the graph hold true in the data.

print(cm.refute_graph(k=2))

Conditional Independence Testing: We are examining whether the data supports the independencies predicted by our DAG. For example, in a simple DAG where A causes B and B causes C, we would expect A to be independent of C given B. This test checks such independencies up to a specified conditioning set size (in this case, 2).

Output Explanation:

Number of Conditional Independencies Entailed by Model: This tells us how many conditional independencies our DAG implies.
Number of Independencies Satisfied by Data: This shows how many of these independencies are actually observed in the data.
Test Passed: This final result indicates whether the overall set of conditional independencies is supported by the data. If the test fails, it suggests that our DAG might not be an accurate representation of the data-generating process.

Identifying the Causal Effect

Next, we identify the causal effect of the treatment (D) on the outcome (Y). This step involves using the DAG to find appropriate adjustment sets that help control for confounders and mediators.

identified_estimand = cm.identify_effect(proceed_when_unidentifiable=True)
print(identified_estimand)

Identification of Causal Effect: The method identifies different ways (estimands) to calculate the causal effect. This could include using backdoor adjustment (adjusting for confounders) or front-door adjustment (using mediators). In this case, it identifies that front-door adjustment via mediator M is suitable for estimating the effect of D on Y.

Output Explanation:

Backdoor Adjustment: It provides an expression involving conditioning on variables like X2 and Z2.
Front-Door Adjustment: It shows how to use the mediator M to estimate the effect of D on Y, which involves understanding how D affects M and how M affects Y.

Estimating the Causal Effect

Finally, we estimate the causal effect using the identified method. Here, we’re using a two-stage regression approach as part of the front-door adjustment strategy.

estimate = cm.estimate_effect(identified_estimand, method_name='frontdoor.two_stage_regression')

Estimation Process: This step involves calculating the actual causal effect by leveraging the front-door criterion. The method accounts for the mediator M to understand the impact of D on Y. Essentially, it first models how D influences M, and then models how M (along with D) influences Y.

Output Explanation: The output provides the estimated average treatment effect (ATE), which quantifies the effect of the treatment D on the outcome Y. This estimate is derived by considering the mediated pathways and ensuring that the effect is not confounded by other variables.

Summary

Causal Graph Definition: Encodes our assumptions about the causal relationships between variables.
Model Visualization: Helps verify and understand the structure of the DAG.
Conditional Independence Testing: Ensures the implied independencies hold in the data, validating the DAG.
Effect Identification: Determines the appropriate method to estimate causal effects based on the DAG structure.
Effect Estimation: Quantifies the causal effect using the identified adjustment strategy, ensuring accurate and robust causal inference.

By following these steps, we can confidently use DAGs and SEMs to understand and estimate causal relationships in complex systems, ensuring that our models are both theoretically sound and empirically validated.

3 June, 2024 ahmed.ismail2013

Ahmed Dawoud