RuntimeError: Input type and weight type should be the same in PyTorch
Cast input tensors to match the neural network weights dtype using x.to(dtype=layer.weight.dtype) or x.float().
Root Cause Analysis
This error occurs when Python tries to pass an input tensor into a PyTorch layer (such as nn.Linear or nn.Conv2d) whose internal weight parameters have a different scalar floating-point precision (such as torch.float64 vs torch.float32).
1. PyTorch Scalar Precision Rules
PyTorch layers default to single precision (torch.float32 / FloatTensor). If a model is converted to double precision using model.double() or half precision with model.half(), all internal weights and biases are converted to that dtype. PyTorch compute kernels require input data and weights to share identical precision to prevent unintended silent precision truncation.
2. Common Causes in Data Ingestion
NumPy arrays default to float64 on 64-bit platforms. When converting a NumPy array to PyTorch via torch.from_numpy(arr), the resulting tensor has dtype torch.float64 (DoubleTensor). Passing this tensor into a standard torch.float32 model triggers RuntimeError: expected scalar type Double but found Float or Input type and weight type should be the same.
3. Resolving Precision Mismatches
Always convert input batches explicitly in the DataLoader or training loop using x = x.float() or x.to(model.weight.dtype).
4. Automatic Mixed Precision (AMP)
When using mixed precision training (torch.cuda.amp.autocast), let PyTorch handle precision casting automatically rather than manually casting weights.
Reproduction Code (MCVE)
import torch
import torch.nn as nn
# Layer with Double weights (float64)
layer = nn.Linear(2, 2).double()
# Input tensor with Float type (float32)
x = torch.tensor([[1.0, 2.0]], dtype=torch.float32)
# RuntimeError: expected scalar type Double but found Float
output = layer(x)
Solution 1: Cast Input Tensor to Match Layer Precision
Cast the input tensor to torch.float64 (.double()) to match the layer weight precision.
import torch
import torch.nn as nn
layer = nn.Linear(2, 2).double()
# Pass matching float64 input
x = torch.tensor([[1.0, 2.0]], dtype=torch.float64)
output = layer(x)
print('Forward pass with matching dtype:', output)
Solution 2: Align Input Dtype Dynamically from Model Parameter
Query the model's weight dtype dynamically so your pipeline adapts seamlessly to float32, float64, or float16.
import torch
import torch.nn as nn
layer = nn.Linear(2, 2)
raw_input = torch.tensor([[1.0, 2.0]], dtype=torch.float64)
# Dynamically cast to layer weight dtype
aligned_input = raw_input.to(dtype=layer.weight.dtype)
output = layer(aligned_input)
print('Dynamic cast forward output:', output.shape)
A common trap is loading CSV data with pd.read_csv() or np.loadtxt(), which defaults to float64, and feeding torch.from_numpy() directly into a standard neural network. Always call torch.tensor(..., dtype=torch.float32) or tensor.float() during dataset preprocessing. Contrast scalar precision mismatches with device mismatches (Expected all tensors to be on the same device). When implementing custom autograd Functions, verify that gradient outputs match the precision of input tensors in the backward pass.