Knowledge Base | Infortrend

Abstract

This document provides step-by-step instructions for configuring NVMe-oF on Linux with Infortrend storage. The setup includes NVMe module loading, multipath settings, and connection validation. This guide ensures a seamless setup process for optimal performance.

Verify NVMe Packages are Installed
Before proceeding, ensure that the required NVMe utilities and dependencies are installed on your system.
- Check if nvme-cli is installed:
```
rpm -q nvme-cli  # For RHEL-based systems
dpkg -l | grep nvme-cli  # For Debian-based systems
```
- If not installed, install nvme-cli:
```
sudo yum install -y nvme-cli  # RHEL-based systems
sudo apt-get install -y nvme-cli  # Debian-based systems
```
- Verify that the NVMe modules exist in the system:
```
lsmod | grep nvme
```
- If no output is returned, manually load the required NVMe modules:
```
modprobe nvme
modprobe nvme-rdma
```
- Verify the modules are successfully loaded:
```
lsmod | grep nvme
```
Load NVMe Modules Automatically
Before connecting to NVMe-oF storage, ensure that the required kernel modules are loaded.
- Create a module configuration file to ensure the modules load automatically at boot:
```
sudo vi /etc/sysconfig/modules/nvme.modules
```
- Add the following lines:
```
#!/bin/sh
modprobe nvme
modprobe nvme-rdma
```
- Set appropriate permissions:
```
chmod 755 /etc/sysconfig/modules/nvme.modules
```
- Verify the module file exists:
```
ls -al /etc/sysconfig/modules/
```
Disable Native NVMe MPIO
Linux native NVMe MPIO can conflict with Infortrend's ALUA-based multipath settings. It should be disabled to ensure proper MPIO functionality.
- Create or modify the module configuration file:
```
sudo nano /etc/modprobe.d/nvme_core.conf
```
- Add the following line to disable NVMe multipath:
```
options nvme_core multipath=N
```
- Rebuild the initramfs to apply changes:
```
sudo dracut -f
```
- Reboot the system:
```
sudo reboot
```
- Verify the configuration:
```
cat /sys/module/nvme_core/parameters/multipath
```

Connect NVMe-oF to Infortrend Storage

Discover NVMe-oF target:
```
nvme discover -t rdma -a 10.0.0.30
```

Connect to the target:

nvme connect-all -t rdma -a 10.0.0.30 -l 50000 -i 4

Verify the connection:
```
nvme list -v
nvme list-subsys
```
Disconnect NVMe-oF:
```
nvme disconnect-all
```

Configure MPIO Using Device Mapper Multipath
Modify the multipath configuration file to optimize MPIO settings for Infortrend RAID systems.
Modify /etc/multipath.conf:

devices {
    device {
        vendor "IFT"
        product ".*"
        path_selector "queue-length 0"
        path_grouping_policy group_by_prio
        prio alua
        path_checker tur
        rr_min_io 100
        rr_weight uniform
        failback immediate
        no_path_retry 20
        hardware_handler "0"
    }
    device {
        vendor "NVME"
        product ".*"
        path_selector "queue-length 0"
        path_grouping_policy group_by_prio
        prio "ANA"
        path_checker "none"
        rr_min_io 100
        rr_weight uniform
        failback immediate
        no_path_retry 3000
        hardware_handler "0"
    }
}

Restart and Enable MPIO Service
- Enable multipathd to start on boot:
```
systemctl enable multipathd
```
- Restart the Multipath Daemon:
```
service multipathd restart
```
- Reload multipath configuration to apply changes:
```
multipath -r
```
- Verify MPIO Configuration:
```
multipath -ll
```

Conclusion

By following these steps, you can successfully configure Linux NVMe-oF with Infortrend storage. Proper NVMe package verification, module loading, and MPIO settings ensure optimized performance and stability.

If you encounter any issues, please contact Infortrend Technical Support.

For additional support, visit: Infortrend Support Portal