imported>DrOwl (New page: nxge = nVidia Gigabit Ethernet ==Numbering== Interfaces are numbered 0,1,2,3 with 3 being nearest the PCI slot. ==Driver== In Solaris 10 U4 and above apparently, otherwise [[:Image:Sun...) |
imported>DrOwl m (Nxge interface moved to nxge interface) |
(No difference)
|
Latest revision as of 17:10, 30 April 2008
nxge = nVidia Gigabit Ethernet
Numbering[edit]
Interfaces are numbered 0,1,2,3 with 3 being nearest the PCI slot.
Driver[edit]
In Solaris 10 U4 and above apparently, otherwise here or from http://www.sun.com/download/products.xml?id=45a593ce.
On U3 or below you must update to Jumbo Kernel Patch 118833-36 or later, reboot immediately following the patch and before applying any more patches.
For SPARC systems, change to the following directory:
cd 10_GigabitEthernet/Solaris_10/sparc/Packages
For sun4v systems, install the software packages by typing the following at the command line:
/usr/sbin/pkgadd -d . SUNWnxge.v SUNWnxgem
For sun4u systems, install the software packages by typing the following at the command line:
/usr/sbin/pkgadd -d . SUNWnxge.u SUNWnxgem
Docs[edit]
http://docs.sun.com/source/820-0115-10/ (for Sun x8 Express Quad Gigabit Ethernet UTP Low Profile Adapter)
Configuration[edit]
Use /kernel/drv/nxge.conf. Here's an example
# #---------------Link Configuration ---------------------- # The link parameters depend on the type of the card # and the port. # 10 gigabit related parameters ( i.e adv_10gfdx_cap) # apply only to 10gigabit ports. # Half duplex is not supported on any NIU card. # # adv-autoneg-cap # Advertise auto-negotiation capability. # default is 1 # adv-autoneg-cap = 1; # # adv_10gfdx_cap # Advertise 10gbps Full duplex capability. # default is 1 # adv_10gfdx_cap = 1; # # adv_1000fdx_cap # Advertise 1gbps Full duplex capability. # default is 1 # adv_1000fdx_cap = 1; # # adv_100fdx_cap # Advertise 100mbps Full duplex capability. # default is 1 # adv_100fdx_cap = 1; # # adv_10fdx_cap # Advertise 10mbps Full duplex capability. # default is 1 # adv_10fdx_cap = 1; # # adv_asmpause_cap # Advertise Asymmetric pause capability. # default is 0 # adv_asmpause_cap = 0; # # adv_pause_cap # Advertise pause capability. # default is 1 # adv_pause_cap = 1; # # #------- Jumbo frame support --------------------------------- # To enable jumbo support for all nxge interfaces, # accept_jumbo = 1; # # To disable jumbo support for all nxge interfaces, # accept_jumbo = 0; # # Default is 0. See the example at the end of this file for # enabling or disabling jumbo for a particular nxge interface. # # #------- Receive DMA Configuration ---------------------------- # # rxdma-intr-time # Interrupts after this number of NIU hardware ticks have # elapsed since the last packet was received. # A value of zero means no time blanking (Default = 8). # # rxdma-intr-pkts # Interrupt after this number of packets have arrived since # the last packet was serviced. A value of zero indicates # no packet blanking (Default = 20). # # Default Interrupt Blanking parameters. # # rxdma-intr-time = 8; # rxdma-intr-pkts = 20; # # #------- Classification and Load Distribution Configuration ------ # # class-opt-****-*** # These variables define how each IP class is configured. # Configuration options range from whether TCAM lookup ie # is enabled to flow hash generation. # This parameters also control how the flow template is # constructed and how packet is distributed within RDC # groups. # # supported classes: # class-opt-ipv4-tcp class-opt-ipv4-udp class-opt-ipv4-sctp # class-opt-ipv4-ah class-opt-ipv6-tcp class-opt-ipv6-udp # class-opt-ipv6-sctp class-opt-ipv6-ah # # Configuration bits (The following bits will be decoded # by the driver as hex format). # # 0010: use MAC Port (for flow key) # 0020: use L2DA (for flow key) # 0040: use VLAN (for flow key) # 0080: use proto (for flow key) # 0100: use IP src addr (for flow key) # 0200: use IP dest addr (for flow key) # 0400: use Src Port (for flow key) # 0800: use Dest Port (for flow key) # # class-opt-ipv4-tcp = fe0; #
See the linked documentation for details on how to specify individual port details.
You can view current interface speeds like this:
root@host # dladm show-dev e1000g0 link: down speed: 0 Mbps duplex: half e1000g1 link: down speed: 0 Mbps duplex: half e1000g2 link: unknown speed: 0 Mbps duplex: unknown e1000g3 link: up speed: 100 Mbps duplex: full nxge0 link: down speed: 0 Mbps duplex: unknown nxge1 link: up speed: 1000 Mbps duplex: full nxge2 link: unknown speed: 0 Mbps duplex: unknown nxge3 link: unknown speed: 0 Mbps duplex: unknown
Performance[edit]
Add this line to /etc/system (use '8' on UltraSparc-T1 systems)
- set ddi_msix_alloc_limit=4
Add this line to a ndd start up script somewhere (set to 16 on UltraSparc-T1 systems)
- ndd set ddi_msix_alloc_limit=4
Firmware[edit]
Updating the firmware on these cards is one scary-arse piece of work!
You get the firmware and driver pack from http://www.sun.com/download/products.xml?id=45a593ce (this is the same line as above on this page). You need to unzip the HTML instructions from 10_GigabitEthernet/Firmware and copy them somewhere you can point a browser at (here). Follow the instructions VERY carefully!
I've written a function to generate the firmware commands from the OS of a server as it's easy to get things wrong, escpeially if you have two or more nxge cards. However, I strongly recommend you verify it's output with what you'd expect before proceeding as I don't want to be held responsible for shafting an expensive server!
nxge_fw_upgrade_cmds() { echo load-base dload NXGE_OBP.FC echo load-base 1 byte-load echo load net:,QGCEM3.12 for i in 0 4 8 12 16; do test -e /dev/nxge$i || continue ifconfig nxge$i|nawk '/ether/{print $2,"set-mac-addr"}'|sed 's/:/ /g' echo "\" `ls -l /dev/nxge$i|awk '{print $NF}'|\ sed 's|../devices||;s/:.*//'`\" begin-select-dev" echo load-base update done echo reset-all }