遮擋、梯度、Integrated Gradients 與 LRP

drop_i = f(x) − f(把第i個特徵換成指定值後的x)

f(x1,x2) = x1 + 2x2 + x1x2
目標 (1,1)：分數4
只把x1換成0：(0,1)，分數2，drop1=2
只把x2換成0：(1,0)，分數1，drop2=3

路徑：gamma(alpha) = b + alpha × (x−b)，alpha從0到1

IG_i = (x_i−b_i) × integral from0to1:
                       ∂f(b+alpha×(x−b)) / ∂x_i  d alpha

s = x1 + 2x2
f(x1,x2) = ReLU(s) − ReLU(s−1)

s≤0：f=0
0<s<1：f=s
s≥1：f=1

IG1 = 1 × [1 × (1/3)] = 1/3
IG2 = 1 × [2 × (1/3)] = 2/3
總和 = 1 = f(1,1)−f(0,0)

sum_i IG_i = f(x)−f(b)

路徑 = (0.1+0.9alpha, alpha)
s(alpha) = 0.1+2.9alpha
到alpha=9/29時，s=1，此後飽和

IG1 = 0.9 × 9/29 = 81/290 ≈ 0.279310345
IG2 = 2 × 9/29 = 18/29 ≈ 0.620689655
總和 = 0.9

z_ij = a_i × w_ij
R_(i←j) = z_ij / sum_k z_kj × R_j
R_i = sum_j R_(i←j)

input = (1,2)
h1 = 1×1 + 2×1 = 3
h2 = 1×2 + 2×1 = 4
output = 2h1 + 1h2 = 10

R_(i←j) = z_ij / (sum_k z_kj + epsilon) × R_j

python3 explanation_mechanics.py

#!/usr/bin/env python3
"""Hand-specified explanation mechanics; Python standard library only.

No trained model, dataset, autograd framework, or attribution package is used.
Saturated-score comparisons use one function. LRP uses a separate, explicitly
positive, bias-free network. They are not a method-quality benchmark.
"""

from fractions import Fraction as F
import json
import math


def saturated_score(x1, x2):
    s = x1 + 2 * x2
    return max(s, 0) - max(s - 1, 0)


def saturated_gradient(x1, x2):
    """Analytic derivative off s=0,1; choose zero at the two kinks.

    The chosen midpoint grids below never hit either kink. This convention
    does not assert differentiability at the kinks or emulate an autograd API.
    """
    s = x1 + 2 * x2
    slope = 1.0 if 0.0 < s < 1.0 else 0.0
    return slope, 2.0 * slope


def midpoint_ig(baseline, target, steps):
    delta = tuple(t - b for b, t in zip(baseline, target))
    gradients = []
    for i in range(steps):
        alpha = (i + 0.5) / steps
        point = tuple(b + alpha * d for b, d in zip(baseline, delta))
        # Only this fixed toy/grid scope is supported by this script.
        s = point[0] + 2 * point[1]
        assert not math.isclose(s, 0.0, abs_tol=1e-12)
        assert not math.isclose(s, 1.0, abs_tol=1e-12)
        gradients.append(saturated_gradient(*point))
    attribution = tuple(delta[j] * sum(g[j] for g in gradients) / steps for j in range(2))
    output_change = saturated_score(*target) - saturated_score(*baseline)
    return {"steps": steps, "attribution": attribution,
            "sum": sum(attribution), "output_change": output_change,
            "completeness_delta": sum(attribution) - output_change}


def central_difference(point, coordinate, step=1e-6):
    above, below = list(point), list(point)
    above[coordinate] += step
    below[coordinate] -= step
    return (saturated_score(*above) - saturated_score(*below)) / (2 * step)


def positive_messages(inputs, weights, relevance, epsilon=F(0)):
    """Fixed positive no-bias z-rule, or positive-denominator epsilon variant.

    epsilon is added at every redistribution layer when requested. No
    residual is redistributed. This is not a general signed/bias LRP API.
    """
    if epsilon < 0 or any(v < 0 for v in (*inputs, *weights)):
        raise ValueError("This toy requires nonnegative inputs/weights/epsilon")
    terms = tuple(a * w for a, w in zip(inputs, weights))
    denominator = sum(terms, F(0)) + epsilon
    if denominator <= 0:
        raise ValueError("Positive nonzero denominator required")
    return tuple(term / denominator * relevance for term in terms)


def two_layer_lrp(epsilon):
    x = (F(1), F(2))
    # Rows correspond to the two hidden units. ReLU leaves both positive h unchanged.
    hidden_weights = ((F(1), F(1)), (F(2), F(1)))
    output_weights = (F(2), F(1))
    hidden = tuple(sum((a * w for a, w in zip(x, row)), F(0)) for row in hidden_weights)
    output = sum((h * w for h, w in zip(hidden, output_weights)), F(0))
    hidden_r = positive_messages(hidden, output_weights, output, epsilon)
    input_messages = tuple(positive_messages(x, row, r, epsilon)
                           for row, r in zip(hidden_weights, hidden_r))
    input_r = tuple(sum((messages[j] for messages in input_messages), F(0)) for j in range(2))
    return {"epsilon_at_each_layer": str(epsilon), "x": list(map(str, x)),
            "hidden_weights_rows": [list(map(str, row)) for row in hidden_weights],
            "hidden_activations": list(map(str, hidden)),
            "output_weights": list(map(str, output_weights)), "output": str(output),
            "hidden_relevance": list(map(str, hidden_r)),
            "input_messages_by_hidden": [list(map(str, row)) for row in input_messages],
            "input_relevance": list(map(str, input_r)),
            "input_relevance_float": list(map(float, input_r)),
            "input_relevance_sum": str(sum(input_r, F(0))),
            "output_minus_input_relevance_sum": str(output - sum(input_r, F(0)))}


def main():
    baseline, target = (0.0, 0.0), (1.0, 1.0)
    gradient = saturated_gradient(*target)
    ig = midpoint_ig(baseline, target, 300)
    score = saturated_score(*target)
    occlusion = tuple(score - saturated_score(*(0.0 if j == i else target[j] for j in range(2)))
                      for i in range(2))
    convergence = [midpoint_ig(baseline, target, n) for n in (8, 32, 128, 512, 2048)]
    nonzero_baseline = (0.1, 0.0)
    changed_baseline_ig = midpoint_ig(nonzero_baseline, target, 300)
    # s(alpha)=1/10+(29/10)alpha crosses 1 at alpha=9/29.
    analytic_nonzero = (F(81, 290), F(18, 29))
    fd_checks = []
    for point in ((0.25, 0.125), target):
        analytic = saturated_gradient(*point)
        finite = tuple(central_difference(point, j) for j in range(2))
        assert all(abs(a - f) < 1e-8 for a, f in zip(analytic, finite))
        fd_checks.append({"point": point, "analytic": analytic, "central_difference": finite})

    def interaction(x1, x2):
        return x1 + 2 * x2 + x1 * x2

    original = interaction(1, 1)
    interaction_drops = (original - interaction(0, 1), original - interaction(1, 0))
    one_layer_x, one_layer_w = (F(3), F(1)), (F(2), F(1))
    one_layer_r = positive_messages(one_layer_x, one_layer_w, F(7))
    z_rule, epsilon_rule = two_layer_lrp(F(0)), two_layer_lrp(F(1, 2))

    assert gradient == (0.0, 0.0) and occlusion == (0.0, 0.0)
    assert ig["attribution"] == (1 / 3, 2 / 3)
    assert abs(ig["completeness_delta"]) < 1e-12
    assert abs(convergence[-1]["completeness_delta"]) < abs(convergence[0]["completeness_delta"])
    assert abs(sum(analytic_nonzero) - F(9, 10)) == 0
    assert all(abs(a - float(b)) < 0.001 for a, b in zip(changed_baseline_ig["attribution"], analytic_nonzero))
    assert interaction_drops == (2, 3) and sum(interaction_drops) == 5 > original
    assert one_layer_r == (F(6), F(1))
    assert z_rule["hidden_relevance"] == ["6", "4"]
    assert z_rule["input_relevance"] == ["4", "6"]
    assert z_rule["input_relevance_sum"] == "10"
    assert epsilon_rule["input_relevance"] == ["4400/1323", "6560/1323"]
    assert F(epsilon_rule["input_relevance_sum"]) < F(epsilon_rule["output"])

    result = {
        "scope": "hand-set scalar mechanisms, not training/real-image attribution/causal evidence or a package benchmark",
        "saturated_function": "ReLU(x1+2*x2)-ReLU(x1+2*x2-1)",
        "saturation_zero_baseline": {"baseline": baseline, "target": target,
            "baseline_score": saturated_score(*baseline), "target_score": score,
            "analytic_target_gradient": gradient,
            "gradient_times_input": tuple(g * x for g, x in zip(gradient, target)),
            "single_feature_zero_occlusion_drops": occlusion,
            "analytic_straight_path_ig": ["1/3", "2/3"], "midpoint_ig": ig},
        "midpoint_resolution_comparison_same_zero_baseline": convergence,
        "nonzero_baseline_same_function_target": {"baseline": nonzero_baseline,
            "target": target, "baseline_score": saturated_score(*nonzero_baseline),
            "path_saturation_alpha": "9/29",
            "analytic_straight_path_ig": list(map(str, analytic_nonzero)),
            "analytic_ig_sum": str(sum(analytic_nonzero)), "midpoint_ig": changed_baseline_ig},
        "off_kink_gradient_central_difference_checks": fd_checks,
        "interaction_occlusion": {"function": "x1+2*x2+x1*x2", "target": [1, 1],
            "score": original, "single_feature_zero_occlusion_drops": interaction_drops,
            "sum_of_drops": sum(interaction_drops), "not_an_additive_decomposition": True},
        "single_positive_linear_z_rule": {"x": [3, 1], "weights": [2, 1],
            "output": 7, "relevance": list(map(str, one_layer_r))},
        "separate_two_layer_positive_no_bias_network": {"z_rule": z_rule,
            "epsilon_variant_no_residual_redistribution": epsilon_rule,
            "scope": "different function from saturated example; all weights/activations positive, no bias; epsilon=1/2 added in both layers"},
        "limits": ["all gradients are explicitly derived, not autograd",
            "straight paths and baselines are hand-chosen; not a causal intervention",
            "300 steps is exactly aligned for the zero-baseline toy; finite quadrature is not generally exact",
            "conservation of a relevance rule does not establish semantic quality or causality"]}
    print(json.dumps(result, ensure_ascii=False, indent=2))


if __name__ == "__main__":
    main()

# DS58 image credits and reuse notes

These illustrations reproduce the source-created diagrams and chart listed below. Their authors, individual grants and modifications are recorded here; no photograph or model/data rerun is included.

## PLOS ONE figures

Sebastian Bach et al. (2015), Figures 2, 3, and 5, “On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation,” *PLOS ONE* 10(7): e0130140, [doi:10.1371/journal.pone.0130140](https://doi.org/10.1371/journal.pone.0130140). The article states [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). The main images are byte-copies of the complete source figures. The mobile derivatives show the complete figure first and stack complete source panels without adding marks, text, or data. The article's separate data-availability statement about some CC0 source images is not being treated as a blanket image grant.

## Integrated Gradients Figure 1

Mukund Sundararajan, Ankur Taly, and Qiqi Yan (2017), Figure 1, “Axiomatic Attribution for Deep Networks,” *Proceedings of the 34th International Conference on Machine Learning*, PMLR 70, pp. 3319–3328, [PMLR article](https://proceedings.mlr.press/v70/sundararajan17a.html). The 2017 ICML author instructions linked the contemporaneous publication-permission form. Its §2 grants CC BY 4.0 and §3 requires citation and a PMLR link. No signed author-specific form copy is claimed. The source figure and caption are cropped from the original PDF page 4; the cover fits the complete crop proportionally on white.

## Distill chart

Pascal Sturmfels, Scott Lundberg, and Su-In Lee (2020), Figure 3 chart, “Visualizing the Impact of Feature Attribution Baselines,” *Distill*, [doi:10.23915/distill.00022](https://doi.org/10.23915/distill.00022). The article-specific footer states that diagrams and text are CC BY 4.0 and excludes externally credited figures. The selected D3 chart is the authors' own visualization and has no separate third-party credit. The currently saved Distill FAQ links CC BY 2.0; this chart relies on the article-specific CC BY 4.0 statement, not on a claim about the FAQ. The selector thumbnails and photograph are excluded. The source's chart plots Goldfinch Softmax output probability against the interpolation constant in one Inception v4 example.

The mobile chart shows the whole chart first, followed by two overlapping x-range crops. The crops omit the long title and x-axis label; readers should use the whole chart for those labels and the full range. No chart values or visual marks were redrawn or changed.