Skip to content

BUG: Compatibility Issue for pytensor on macOS 15.4 #1342

Open
@astrokang

Description

@astrokang

Describe the issue:

After updating to macOS 15.4, I encountered a failure when running pymc with pytensor. The error seems to be related to constant folding during graph optimization and LTO (Link Time Optimization) issues during compilation. I don't know. But I have tested that this issue does not occur on macOS 14.x.

Reproduceable code example:

import numpy as np
import pymc as pm
        

def pymc_fitting(x_obs, y_obs, y_err, if_xerr = False, x_err = 0., 
                 if_intrin = True,
                 x_fit_low = None, x_fit_upp = None, 
                 CL_value = 0.90, random_seed = 12):

    """
    Fitting function with pymc

    Parameters:
    
    - x_obs (array): x-axis observed values
    - y_obs (array): y-axis observed values
    - y_err (array): y-axis error values
    - if_xerr (bool): whether consider x-axis error
    - x_err (array): x-axis error values
    - x_fit_low (float): lower limit of x-axis for fitting
    - x_fit_upp (float): upper limit of x-axis for fitting
    - CL_value (float): confidence level value
    - random_seed (int): random seed value

    Returns:

    [x_fit, y_fit_med, [y_fit_low, y_fit_upp]]
    
    - x_fit (array): x-axis fitting values
    - y_fit_med (array): y-axis fitting values
    - y_fit_low (array): y-axis fitting values (lower limit)
    - y_fit_upp (array): y-axis fitting values (upper limit)
    """
    
    # Defime PyMC model and consider linear fitting ("model" Processed implicitly by PyMC)
    with pm.Model() as model:
        
        # Priors for slope and intercept
        k_value = pm.Uniform("slope", lower = -10, upper = 10)
        b_value = pm.Uniform("intercept", lower = -10, upper = 10)

        if not if_xerr:

            x_true_est = x_obs
            
        else:
            
            # Hidden variable for true x values (considering x-axis parameter error)
            x_true_est = pm.Normal("x_true", mu = x_obs, sigma = x_err, shape = len(x_obs))
        
        # Calculate the expected value of y (fitting function)
        y_est = k_value * x_true_est + b_value

        # Whether consider x-axis error
        if if_intrin:

            sigma_scatter = pm.HalfNormal("sigma_scatter", sigma = 1)

        else:

            sigma_scatter = 0
        
        # Observed y error (Implicit likelihood calculation)
        y_obs_likelihood = pm.Normal("y_obs", mu = y_est, sigma = np.sqrt(y_err**2 + sigma_scatter**2), observed = y_obs)
    
        # Sample from the posterior
        trace = pm.sample(5000, return_inferencedata = True, chains = 4, target_accept = 0.99, random_seed = random_seed)


    k_samples = trace.posterior["slope"].values.flatten()
    b_samples = trace.posterior["intercept"].values.flatten()

    if x_fit_low is None:

        x_fit_low = min(x_obs)
        x_fit_upp = max(x_obs)

    x_fit = np.linspace(x_fit_low, x_fit_upp, 1000)
    
    # Calculate the fitted y values for each sample
    y_fit_samples = np.outer(k_samples, x_fit) + b_samples[:, None]
    
    # Calculate the mean and confidence intervals for the fitted y values 
    y_fit_med = np.percentile(y_fit_samples, 50, axis = 0)
    y_fit_low = np.percentile(y_fit_samples, 50 * (1 - CL_value), axis = 0)
    y_fit_upp = np.percentile(y_fit_samples, 50 * (1 + CL_value), axis = 0)
    

    return [x_fit, y_fit_med, [y_fit_low, y_fit_upp]]




# Random seed   
np.random.seed(12)

# True line
k_true = 2
b_true = -5

x_true = np.linspace(0, 10, 50)
y_true = k_true * x_true + b_true

x_plot = np.linspace(-1, 11, 50)
ax_main.plot(x_plot, k_true * x_plot + b_true, label = label_list[0],
             color = 'k', linewidth = 10, linestyle = '-', zorder = 5)

# Observed data
x_err = np.random.uniform(0.2, 0.6, size = len(x_true))
x_obs = x_true + np.random.normal(size = len(x_true)) * x_err

intrin_sig = 2        # Intrinsic scatter
y_err = np.random.uniform(1, 3, size = len(x_true))
y_obs = y_true + np.random.normal(size = len(x_true)) * np.sqrt(y_err**2 + intrin_sig**2)


# pymc fitting
# interval_list = [x_fit, y_fit_mean, [y_fit_low, y_fit_upp]]
interval_list = pymc_fitting(x_obs, y_obs, y_err, if_xerr = True, x_err = x_err, 
                             CL_value = 0.90, random_seed = 12)

Error message:

ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: ExpandDims{axis=0}(2)
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/graph/rewriting/basic.py", line 1916, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/graph/rewriting/basic.py", line 1086, in transform
    return self.fn(fgraph, node)
           ~~~~~~~^^^^^^^^^^^^^^
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/tensor/rewriting/basic.py", line 1173, in constant_folding
    return unconditional_constant_folding.transform(fgraph, node)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/graph/rewriting/basic.py", line 1086, in transform
    return self.fn(fgraph, node)
           ~~~~~~~^^^^^^^^^^^^^^
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/tensor/rewriting/basic.py", line 1122, in unconditional_constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/link/c/op.py", line 119, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/link/c/op.py", line 84, in make_c_thunk
    outputs = cl.make_thunk(
        input_storage=node_input_storage, output_storage=node_output_storage
    )
  File "/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/link/c/basic.py", line 1185, in make_thunk
...
ld: -lto_library library filename must be 'libLTO.dylib'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)


Output is truncated. View as a [scrollable element](command:cellOutput.enableScrolling?87be159b-d347-49e3-8226-abc1672dda64) or open in a [text editor](command:workbench.action.openLargeOutput?87be159b-d347-49e3-8226-abc1672dda64). Adjust cell output [settings](command:workbench.action.openSettings?%5B%22%40tag%3AnotebookOutputLayout%22%5D)...

You can find the C code in this temporary file: [/var/folders/ch/qk2fyfsd5f75z607yqb892200000gn/T/pytensor_compilation_error_xiy08s1v](https://file+.vscode-resource.vscode-cdn.net/var/folders/ch/qk2fyfsd5f75z607yqb892200000gn/T/pytensor_compilation_error_xiy08s1v)
library to_library is not found.

......

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/link/c/lazylinker_c.py:66
     65         if version != actual_version:
---> 66             raise ImportError(
     67                 "Version check of the existing lazylinker compiled file."
     68                 f" Looking for version {version}, but found {actual_version}. "
     69                 f"Extra debug information: force_compile={force_compile}, _need_reload={_need_reload}"
     70             )
     71 except ImportError:

ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.3, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
File ~/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/link/c/lazylinker_c.py:87
     86     if version != actual_version:
---> 87         raise ImportError(
     88             "Version check of the existing lazylinker compiled file."
     89             f" Looking for version {version}, but found {actual_version}. "
     90             f"Extra debug information: force_compile={force_compile}, _need_reload={_need_reload}"
     91         )
     92 except ImportError:
     93     # It is useless to try to compile if there isn't any
...
CompileError: Compilation failed (return status=1):
/Users/dendrophile/miniconda3/envs/pymc_env/bin/clang++ -dynamiclib -g -Wno-c++11-narrowing -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -fPIC -undefined dynamic_lookup -ld64 -I/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/numpy/_core/include -I/Users/dendrophile/miniconda3/envs/pymc_env/include/python3.13 -I/Users/dendrophile/miniconda3/envs/pymc_env/lib/python3.13/site-packages/pytensor/link/c/c_code -L/Users/dendrophile/miniconda3/envs/pymc_env/lib -fvisibility=hidden -o /Users/dendrophile/.pytensor/compiledir_macOS-15.4-arm64-arm-64bit-Mach-O-arm-3.13.2-64/lazylinker_ext/lazylinker_ext.so /Users/dendrophile/.pytensor/compiledir_macOS-15.4-arm64-arm-64bit-Mach-O-arm-3.13.2-64/lazylinker_ext/mod.cpp
ld: -lto_library library filename must be 'libLTO.dylib'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

PyMC version information:

Environment
• macOS version: 15.4
• Python version: 3.13 (inside Conda environment)
• pytensor version: (latest from conda-forge)
• pymc version: 5.21.1

Context for the issue:

I have successfully run my code just a week before, but now it has failed, just after I updated my macOS to 15.4 yesterday. Tested on macOS 14 (inside a VM) and everything worked fine, confirming that the issue is specific to macOS 15.4.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions