Skip to content

Support NN models with multiple inputs #161

Open
@sashuIya

Description

@sashuIya

Additionally to the sequence, we'd like to provide some other input (of some different size) to the model. A simple basic example to illustrate:

class SimpleConv(nn.Module):
    def __init__(self):
        self.conv_net = nn.Sequential(
            nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1),
            nn.ReLU(inplace=True),
        )

        self.fc_net = nn.Sequential(
            nn.Linear(channels_in, channels_out),
            nn.ReLU(inplace=True),
        )
 

    def forward(self, x: List[np.ndarray]):
        y1 = self.conv_net(x[0])
        y2 = self.fc_net(x[1])
        y = torch.cat((y1, y2), 1)
        return y

Do you think we could modify the _get_batch() function to return a tuple(List[np.ndarray], np.ndarray)?
https://github.com/FunctionLab/selene/blob/master/selene_sdk/train_model.py#L346-L355

Maybe we could wrap the
https://github.com/FunctionLab/selene/blob/master/selene_sdk/train_model.py#L453-L464
into some function, which will return either a single Tensor or a List[Tensor] for the inputs, based on the provided inputs type? Or would be there a better design solution?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions