Content added Content deleted
imported>DrOwl m (Nxge interface moved to nxge interface) |
imported>DrOwl (Nxge interface moved to nxge interface) |
||
| Line 1: | Line 1: | ||
#REDIRECT [[nxge interface]] |
|||
nxge = nVidia Gigabit Ethernet |
|||
==Numbering== |
|||
Interfaces are numbered 0,1,2,3 with 3 being nearest the PCI slot. |
|||
==Driver== |
|||
In Solaris 10 U4 and above apparently, otherwise [[:Image:Sun_10_Gigabit_Ethernet_driver.zip|here]] or from http://www.sun.com/download/products.xml?id=45a593ce. |
|||
On U3 or below you must update to Jumbo Kernel Patch 118833-36 or later, reboot immediately following the patch and before applying any more patches. |
|||
For SPARC systems, change to the following directory: |
|||
cd 10_GigabitEthernet/Solaris_10/sparc/Packages |
|||
For <strong>sun4v</strong> systems, install the software packages by typing the following at the command line: |
|||
/usr/sbin/pkgadd -d . SUNWnxge.v SUNWnxgem |
|||
For <strong>sun4u</strong> systems, install the software packages by typing the following at the command line: |
|||
/usr/sbin/pkgadd -d . SUNWnxge.u SUNWnxgem |
|||
==Docs== |
|||
http://docs.sun.com/source/820-0115-10/ (for Sun x8 Express Quad Gigabit Ethernet UTP Low Profile Adapter) |
|||
==Configuration== |
|||
Use /kernel/drv/nxge.conf. Here's an example |
|||
<pre> |
|||
# |
|||
#---------------Link Configuration ---------------------- |
|||
# The link parameters depend on the type of the card |
|||
# and the port. |
|||
# 10 gigabit related parameters ( i.e adv_10gfdx_cap) |
|||
# apply only to 10gigabit ports. |
|||
# Half duplex is not supported on any NIU card. |
|||
# |
|||
# adv-autoneg-cap |
|||
# Advertise auto-negotiation capability. |
|||
# default is 1 |
|||
# adv-autoneg-cap = 1; |
|||
# |
|||
# adv_10gfdx_cap |
|||
# Advertise 10gbps Full duplex capability. |
|||
# default is 1 |
|||
# adv_10gfdx_cap = 1; |
|||
# |
|||
# adv_1000fdx_cap |
|||
# Advertise 1gbps Full duplex capability. |
|||
# default is 1 |
|||
# adv_1000fdx_cap = 1; |
|||
# |
|||
# adv_100fdx_cap |
|||
# Advertise 100mbps Full duplex capability. |
|||
# default is 1 |
|||
# adv_100fdx_cap = 1; |
|||
# |
|||
# adv_10fdx_cap |
|||
# Advertise 10mbps Full duplex capability. |
|||
# default is 1 |
|||
# adv_10fdx_cap = 1; |
|||
# |
|||
# adv_asmpause_cap |
|||
# Advertise Asymmetric pause capability. |
|||
# default is 0 |
|||
# adv_asmpause_cap = 0; |
|||
# |
|||
# adv_pause_cap |
|||
# Advertise pause capability. |
|||
# default is 1 |
|||
# adv_pause_cap = 1; |
|||
# |
|||
# |
|||
#------- Jumbo frame support --------------------------------- |
|||
# To enable jumbo support for all nxge interfaces, |
|||
# accept_jumbo = 1; |
|||
# |
|||
# To disable jumbo support for all nxge interfaces, |
|||
# accept_jumbo = 0; |
|||
# |
|||
# Default is 0. See the example at the end of this file for |
|||
# enabling or disabling jumbo for a particular nxge interface. |
|||
# |
|||
# |
|||
#------- Receive DMA Configuration ---------------------------- |
|||
# |
|||
# rxdma-intr-time |
|||
# Interrupts after this number of NIU hardware ticks have |
|||
# elapsed since the last packet was received. |
|||
# A value of zero means no time blanking (Default = 8). |
|||
# |
|||
# rxdma-intr-pkts |
|||
# Interrupt after this number of packets have arrived since |
|||
# the last packet was serviced. A value of zero indicates |
|||
# no packet blanking (Default = 20). |
|||
# |
|||
# Default Interrupt Blanking parameters. |
|||
# |
|||
# rxdma-intr-time = 8; |
|||
# rxdma-intr-pkts = 20; |
|||
# |
|||
# |
|||
#------- Classification and Load Distribution Configuration ------ |
|||
# |
|||
# class-opt-****-*** |
|||
# These variables define how each IP class is configured. |
|||
# Configuration options range from whether TCAM lookup ie |
|||
# is enabled to flow hash generation. |
|||
# This parameters also control how the flow template is |
|||
# constructed and how packet is distributed within RDC |
|||
# groups. |
|||
# |
|||
# supported classes: |
|||
# class-opt-ipv4-tcp class-opt-ipv4-udp class-opt-ipv4-sctp |
|||
# class-opt-ipv4-ah class-opt-ipv6-tcp class-opt-ipv6-udp |
|||
# class-opt-ipv6-sctp class-opt-ipv6-ah |
|||
# |
|||
# Configuration bits (The following bits will be decoded |
|||
# by the driver as hex format). |
|||
# |
|||
# 0010: use MAC Port (for flow key) |
|||
# 0020: use L2DA (for flow key) |
|||
# 0040: use VLAN (for flow key) |
|||
# 0080: use proto (for flow key) |
|||
# 0100: use IP src addr (for flow key) |
|||
# 0200: use IP dest addr (for flow key) |
|||
# 0400: use Src Port (for flow key) |
|||
# 0800: use Dest Port (for flow key) |
|||
# |
|||
# class-opt-ipv4-tcp = fe0; |
|||
# |
|||
</pre> |
|||
See the linked documentation for details on how to specify individual port details. |
|||
You can view current interface speeds like this: |
|||
<pre> |
|||
root@host # dladm show-dev |
|||
e1000g0 link: down speed: 0 Mbps duplex: half |
|||
e1000g1 link: down speed: 0 Mbps duplex: half |
|||
e1000g2 link: unknown speed: 0 Mbps duplex: unknown |
|||
e1000g3 link: up speed: 100 Mbps duplex: full |
|||
nxge0 link: down speed: 0 Mbps duplex: unknown |
|||
nxge1 link: up speed: 1000 Mbps duplex: full |
|||
nxge2 link: unknown speed: 0 Mbps duplex: unknown |
|||
nxge3 link: unknown speed: 0 Mbps duplex: unknown |
|||
</pre> |
|||
==Performance== |
|||
Add this line to /etc/system (use '8' on UltraSparc-T1 systems) |
|||
* set ddi_msix_alloc_limit=4 |
|||
Add this line to a ndd start up script somewhere (set to 16 on UltraSparc-T1 systems) |
|||
* ndd set ddi_msix_alloc_limit=4 |
|||
==Firmware== |
|||
Updating the firmware on these cards is one scary-arse piece of work! |
|||
You get the firmware and driver pack from http://www.sun.com/download/products.xml?id=45a593ce (this is the same line as above on this page). You need to unzip the HTML instructions from 10_GigabitEthernet/Firmware and copy them somewhere you can point a browser at ([http://10.144.132.60/firmware/nxge/NXGE_OBP.HTM here]). |
|||
<b>Follow the instructions VERY carefully!</b> |
|||
I've written a function to generate the firmware commands from the OS of a server as it's easy to get things wrong, escpeially if you have two or more nxge cards. However, I <b>strongly</b> recommend you verify it's output with what you'd expect before proceeding as I don't want to be held responsible for shafting an expensive server! |
|||
<pre> |
|||
nxge_fw_upgrade_cmds() { |
|||
echo load-base dload NXGE_OBP.FC |
|||
echo load-base 1 byte-load |
|||
echo load net:,QGCEM3.12 |
|||
for i in 0 4 8 12 16; do |
|||
test -e /dev/nxge$i || continue |
|||
ifconfig nxge$i|nawk '/ether/{print $2,"set-mac-addr"}'|sed 's/:/ /g' |
|||
echo "\" `ls -l /dev/nxge$i|awk '{print $NF}'|\ |
|||
sed 's|../devices||;s/:.*//'`\" begin-select-dev" |
|||
echo load-base update |
|||
done |
|||
echo reset-all |
|||
} |
|||
</pre> |
|||
[[Category:UNIX]] |
|||
[[Category:Solaris]] |
|||
[[Category:Solaris Networking]] |
|||
Revision as of 17:10, 30 April 2008
Redirect to: