Scikit-Learn: ImportError: cannot import name _get_column_indices from sklearn.utils

Verified FixPython 3.10+Scikit-Learn 1.4+Silo: scikit-learn

Quick Fix / Solution Rapide

This error occurs when Python tries to import the private helper function _get_column_indices from sklearn.utils, which was removed in recent Scikit-learn releases. Use public ColumnTransformer or make_column_selector from sklearn.compose.

Root Cause Analysis

This error occurs when Python code or third-party legacy packages attempt to import the private utility function _get_column_indices from sklearn.utils or sklearn.utils._indexing after upgrading to Scikit-Learn 1.2+.

Root Cause 1: Removal of Private Underscore APIs in Scikit-Learn Releases

In Python and Scikit-Learn conventions, any function, class, or module prefixed with a leading underscore (such as _get_column_indices) is considered internal private implementation detail. Scikit-Learn maintainers refactored the internal indexing and column resolution architecture in Scikit-Learn 1.2+ and completely removed _get_column_indices in favor of consolidated public selectors.

Root Cause 2: Outdated Third-Party Extensions and Tutorials

Many older blog posts, StackOverflow answers, and unmaintained custom transformer libraries (such as older versions of category_encoders, scikit-lego, or custom feature unions) hardcoded from sklearn.utils import _get_column_indices to extract integer indices from pandas column names.

Root Cause 3: Pinned Dependencies vs Environment Upgrades

When environments are updated with pip install --upgrade scikit-learn without upgrading dependent packages, downstream libraries relying on legacy private functions fail at import time.

Root Cause 4: Transition to Public Column Selection Protocols

Scikit-Learn now provides standard, fully documented column selection utilities through sklearn.compose.make_column_selector and ColumnTransformer, rendering internal index lookups obsolete.

Reproduction Code (MCVE)

Example: Bug Reproduction

# Simulating import of removed private function from sklearn.utils
class MockSklearnUtilsModule:
    """Simulates sklearn.utils in Scikit-Learn 1.2+ where private indexing APIs were removed."""
    __name__ = "sklearn.utils"

utils_module = MockSklearnUtilsModule()

# Attempting to import _get_column_indices triggers ImportError
if not hasattr(utils_module, "_get_column_indices"):
    raise ImportError("cannot import name '_get_column_indices' from 'sklearn.utils'")

Solution 1: Use Scikit-Learn make_column_selector and ColumnTransformer

Use the official make_column_selector and ColumnTransformer from sklearn.compose to handle column transformations by dtype or column name without private utilities.

Example: Recommended Solution

from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.preprocessing import StandardScaler, OneHotEncoder
import pandas as pd

# Create a sample mixed-type dataset
df = pd.DataFrame({
    "age": [25, 45, 30],
    "salary": [50000.0, 95000.0, 62000.0],
    "city": ["Paris", "Tokyo", "London"]
})

# Use public column selectors instead of internal private indexers
preprocessor = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), make_column_selector(dtype_include="number")),
        ("cat", OneHotEncoder(sparse_output=False), make_column_selector(dtype_include="object"))
    ]
)

transformed = preprocessor.fit_transform(df)
print("Transformed feature matrix shape:", transformed.shape)
assert transformed.shape == (3, 5)

Solution 2: Implement a Safe Public Column Index Resolution Helper

If your custom code genuinely needs column index mappings, resolve column indices directly from the DataFrame columns list.

Example: Alternative Solution

import pandas as pd

def get_column_indices_safe(df: pd.DataFrame, target_columns: list[str]) -> list[int]:
    """Public and stable helper to find integer column positions in a DataFrame."""
    col_map = {col_name: idx for idx, col_name in enumerate(df.columns)}
    return [col_map[col] for col in target_columns if col in col_map]

df_sample = pd.DataFrame({"user_id": [1, 2], "revenue": [100, 200], "status": ["A", "B"]})
indices = get_column_indices_safe(df_sample, ["revenue", "status"])

print("Resolved column indices:", indices)
assert indices == [1, 2]

A critical best practice when building production machine learning pipelines with Scikit-Learn is never importing symbols with leading underscores (from sklearn.xxx import _something). Private APIs offer zero semantic versioning guarantees and can change or disappear in minor or patch releases.

Another edge case occurs with sklearn.utils._safe_indexing. Similar to _get_column_indices, _safe_indexing was moved and renamed to sklearn.utils.safe_indexing.

Contrast ImportError with AttributeError: ImportError: cannot import name ... occurs when the module exists but the requested name is missing from the module namespace; ModuleNotFoundError occurs when the top-level module or package itself cannot be located.

Previous Next