API Reference¶

creyone_layer.wrap.wrap_conv(cls, opt=None)[source]¶

Wrap a Conv Nd class with flexible argument parsing and optional behaviors.

Parameters:

cls (type[torch.nn.Conv2d]) – A ConvNd-compatible class to wrap.
opt (set | str | None) – A ‘+’-separated string of option flags: - ‘grid’: use the kernel size arg as the stride (grid-like sampling). - ‘ap’: auto-pad so the output spatial size matches the input. - ‘dw’: set groups = in_channels (depthwise convolution).

Returns:

A factory function that accepts (c1, c2, k, …) positionally or as kwargs and forwards them to cls with stride, padding, dilation, and groups set.

creyone_layer.wrap.wrap_pool(cls, opt=None)[source]¶

Wrap a pooling class with flexible argument parsing and optional behaviors.

Parameters:

cls – A PoolNd-compatible class to wrap. Dilation is forwarded automatically only when the class supports it (MaxPoolNd does, AvgPoolNd does not).
opt (set | str | None) – A ‘+’-separated string of option flags: - ‘grid’: use the kernel size arg as the stride (grid-like sampling). - ‘ap’: auto-pad so the output spatial size matches the input.

Returns:

A factory function that accepts (k, …) positionally or as kwargs and forwards them to cls with stride, padding, and dilation set.

creyone_layer.init.same_as_linear(w, group_dim=None)[source]¶

Initialize a weight tensor the same way nn.Linear initializes its weight.

Parameters:

w (torch.Tensor) – Weight tensor to initialize in-place. Must be 2D, unless group_dim is given, in which case w must be 2D after removing group_dim (i.e. every slice along group_dim is 2D).
group_dim (int) – If given, apply the initialization independently to each slice along this dimension instead of to w as a whole.

creyone_layer.init.init_lora_(x, mode='trunc_', general_init=True, **kwargs)[source]¶

Initialize LoRA adapter weights in-place.

Initializes adwA with Kaiming uniform (general) or truncated/normal distribution, and adwB with zeros (general) or the specified distribution.

Parameters:

x (LoRALinear) – A LoRALinear module whose parameters will be initialized.
mode (str) – Prefix for the torch.nn.init function to use when general_init=False (e.g. 'trunc_' -> trunc_normal_).
general_init (bool) – If True, use Kaiming uniform for adwA and zeros for adwB. If False, apply the distribution specified by mode to both.
**kwargs – Additional keyword arguments forwarded to the init function.

creyone_layer.init.init_linear_(x, mode='trunc_', std=0.02, fan='fan_in', **kwargs)[source]¶

Initialize a linear or conv layer’s weight in-place and zero its bias.

Parameters:

x (torch.nn.Linear | torch.nn.Conv2d) – An nn.Linear or nn.Conv2d module to initialize.
mode (str) – Prefix for the torch.nn.init function ('': normal_, 'trunc_': trunc_normal_, 'kaiming_': kaiming_normal_.)
std (float) – Standard deviation used when mode is '' or 'trunc_'.
fan (str) – Fan mode passed to kaiming_normal_ when mode='kaiming_'.
**kwargs – Additional keyword arguments forwarded to the init function.

creyone_layer.init.apply_init(x, general_init=True, **kwargs)[source]¶

Apply weight initialization to a linear, conv, or LoRA layer in-place.

For LoRALinear, initializes both the base linear weights and the LoRA adapter weights. For plain nn.Linear / nn.Conv2d, only the base weights are initialized. Other module types are silently skipped.

Parameters:

x (torch.nn.Linear | torch.nn.Conv2d) – Module to initialize.
general_init (bool) – Passed to init_lora_ to choose between standard LoRA initialization (Kaiming + zeros) and the custom distribution.
**kwargs – Forwarded to init_linear_ and init_lora_.

creyone_layer.init.init_linear(mode='trunc_', std=0.02, fan='fan_in')[source]¶

Return a callable that initializes a linear/conv/LoRA layer with fixed settings.

Convenient for use with module.apply().

Parameters:

mode (str) – Init mode prefix (see init_linear_).
std (float) – Standard deviation for normal-family initializers.
fan (str) – Fan mode for Kaiming initialization.

Returns:

A partial of apply_init with mode, std, and fan bound.

creyone_layer.init.init_norm_(x, val=1.0)[source]¶

Initialize normalization layer weights to val and biases to zero in-place.

Supports BatchNorm1d, BatchNorm2d, GroupNorm, and LayerNorm. Other module types are silently skipped.

Parameters:

x (torch.nn.Module) – Module to initialize.
val (float) – Constant value for the weight parameter.

creyone_layer.init.init_norm(val=1.0)[source]¶

Return a callable that initializes normalization layers with a fixed weight value.

Convenient for use with module.apply().

Parameters:: val (float) – Constant value to set for the weight parameter.
Returns:: A partial of init_norm_ with val bound.

class creyone_layer.layer_scale.LayerScale(*args, **kwargs)[source]¶

Parameters:

dim (int)
init_values (float)
inplace (bool)

class creyone_layer.linear.conv.Linear2d(*args, **kwargs)[source]¶

Parameters:

args (Any)
kwargs (Any)

Return type:

Any

class creyone_layer.linear.conv.Linear3d(*args, **kwargs)[source]¶

Parameters:

args (Any)
kwargs (Any)

Return type:

Any

Linear layer with Low-Rank Adaptation (LoRA)

Portions based on loralib by Microsoft Corporation (https://github.com/microsoft/LoRA), licensed under the MIT License.

References

Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (ICLR 2022). https://arxiv.org/abs/2106.09685

class creyone_layer.linear.lora.LoRALinear(*args, **kwargs)[source]¶

Linear layer with Low-Rank Adaptation (LoRA).

Augments a frozen nn.Linear with trainable low-rank matrices A and B so that the effective weight becomes W + B @ A. Only A and B are updated during fine-tuning; the base weight is left frozen.

The forward computation is:

h = x @ W.T + (dropout(x) @ A.T @ B.T) * (alpha / r)

Parameters:

in_features (int) – Size of each input sample.
out_features (int) – Size of each output sample.
lora_r (int) – Rank of the low-rank decomposition. When 0 the layer behaves identically to a plain nn.Linear.
alpha (float) – LoRA scaling factor. The effective scale applied to the LoRA contribution is alpha / lora_r.
drop (float) – Dropout probability applied to the input before the LoRA branch.
fan_in_fan_out (bool) – Set True when the base weight is stored as (in_features, out_features) (e.g. GPT-2 Conv1D) instead of the default PyTorch layout.
**kwargs – Additional keyword arguments forwarded to nn.Linear (e.g. bias, device, dtype).

forward(x)[source]¶

Compute the LoRA-augmented linear transformation.

Parameters:: x (torch.Tensor) – Input tensor of shape (..., in_features).
Returns:: Output tensor of shape (..., out_features).
Return type:: torch.Tensor

creyone_layer.utils.registry.layer_entrypoint(layer_name, layer_family=None, module_filter=None)[source]¶

Fetch a model entrypoint for specified model name

Parameters:

layer_name (str)
layer_family (str | None)
module_filter (str | None)

Return type:

Callable[[…], Any]

Layer/Module Helpers

creyone_layer.utils.helpers.ntuple(n)[source]¶

Return a function that converts input to an n-tuple.

Scalar values are repeated n times, while iterables are converted to tuples. Strings are treated as scalars to avoid character-level splitting.

Parameters:: n – Target tuple length.
Returns:: Function that converts input to n-tuple.
Return type:: Callable