Overview

The ThunderX2 Pod 1 consists of 3 Arm®v8 based Gigabyte R281-T91 servers and 1 Ampere HR330A jumphost server with the following specifications per server:


Gigabyte R281-T91Ampere HR330A
CPU2x 28 core 2.0 GHz Marvell TX2 Processors1x 32 core 3.3 GHz Ampere ARMv8 64-bit Processor
RAM4x 32GB RDIMM-Tx2 8x 16GB DDR4-DIMM
Storage1x 480GB SSD
1x 1TB HDD
1x 480GB SSD
Networking2x 10Gbe SFP+ (not connected)
2x 1/10Gbe BASE-T (1 connected)
2x 40Gbe QSFP (coming soon)
1x IPMI / Lights-out Management

2x 10Gbe SFP+ (connected)

1x 1Gbe BASE-T (not connected)

1x IPMI / Lights-out Management

Usage & Users

This information can now be found on the Akraino dashboard. Check the Shared Community Lab page for more information.

Server Access

For the meantime when you request access to the Akraino Pods at UNH-IOL we will ask that you send us your public ssh key so we can add it to the root users of those machines.

If you wish to have your own user we can get that setup, but it is much quicker and easier if you just stick with the root user

IPMI Access

When booking a pod, the Akraino dashboard will provide IPMI credentials via email.

You can access an IPMI interface like the example below:

thunder2-gigabyte1-ipmi.akr.iol.unh.edu

Then enter the username and password provided from the file in /opt and now you can manage the system on your and will even be able to add your own public keys in the event of a reinstall of the machine.

If you have any issues with any of the IPMI interfaces please email akraino-lab@iol.unh.edu and we will assist you in anyway we can.

Networking

The networking within the pod is currently provided by a 10Gbe BASE-T switch, which only supports static configurations (i.e. no SDN control).  This switch will be replaced with an Edgecore 7816-64X that is currently on order, along with 40Gbe NICs to be installed on each server.

NetworkIP NetworkVLAN IDDescription
IPMI / Management

10.11.17.0/24

GW: 10.11.17.1

3017Connections to all IPMI interfaces on each server and the switch management interface
Public

10.11.18.0/24

GW: 10.11.18.1

3018Public network (able to reach the Internet), available to each server.


Servers and Switches

The 3 Gigabyte R281-T91 have 3 sets of dual port interfaces:

  • Mellanox Technologies MT27700 on PCI-e expansion slot - two ports on PCI addresses 0000:91:00.0 and 0000:91:00.0 (enp145s0f0 and enp14s0f1)
  • QLogic Corp. FastLinQ QL41000 Series integrated on the main board - two ports on PCI addresses 0000:0c:00.0 and 0000:0c:00.1 (enp12s0f0 and enp12s0f1 or eno1 and eno2)
  • Intel 10G X550T on PCI-e expansion slot - two ports on PCI address 0000:05:00.0 and 0000:05:00.1 (enp5s0f0 and enp5s0f1); not connected at the moment

In the table below, the switch ports for the 3 gigabyte servers are in the following order:

  • Mellanox MT27700 port 1
  • Mellanox MT27700 port 0
  • QLogic FaslinQ QL41000 port 0
  • QLogic FaslinQ QL41000 port 1


The ampere-jumphost2 has the following network interfaces:

  • Mellanox MT27710 on PCI-e expansion slot - two  ports at PCI addresses 0000:01:00.0 and 0000:01:00.1 (enp1s0f0 and enp1s0f1)
  • Intel Gigabit I210 on PCI-e expansion slowt - one port at PCI address 0002:01:00.0 (enP2p1s0); not connected at the moment

In the table below, the switch ports are in the following order:

  • MT27710 port 1
  • MT27710 port 0


Server NameIPMI AddressIPMI VLAN IDPublic Network AddressPublic Network VLAN IDSwitch Port(s)Serial ConnectionOS Installed
thunder2-gigabyte110.11.17.11301710.11.18.113018gigabyte-edgecore2: left 40Bbe Port 1, right 40Gbe Port 2, left 10Gbe Port 7 Breakout 1,  right 10Gbe Port 7 Breakout 2Async 1/6CentOS 7.6
thunder2-gigabyte210.11.17.12301710.11.18.123018gigabyte-edgecore2: left 40Bbe Port 3, right 40Gbe Port 4, left 10Gbe Port 7 Breakout 3, right 10Gbe Port 7 Breakout 4Async 1/7Ubuntu 18.04
thunder2-gigabyte310.11.17.13301710.11.18.133018gigabyte-edgecore2: left 40Bbe Port 5, right 40Gbe Port 6, left 10Gbe Port 9 Breakout 1,  right 10Gbe Port 9 Breakout 2Async 1/8Ubuntu 18.04
thunder2-jumphost10.11.17.14301710.11.18.143018gigabyte-edgecore2: left 10 Gbe Port 9 breakout 3, right 10 Gbe Port 9 breakout 4N/AUbuntu 18.04
Switch NameMgmt AddressVLAN IDSerial ConnectionOS Installed
thunder2-egdecore10.11.17.53017Async 1/2SONiC
  • No labels