QA Cafe - Xbox Router Compatibility Testing

Application Notes > Xbox Router Compatibility Testing

Xbox Router Compatibility Testing

CDRouter 4.2 introduces an Xbox module with test cases modeled after the Xbox console connectivity tests. While CDRouter's Xbox module is not a replacement for Microsoft's Xbox router certification program, it does allow CDRouter users to incorporate the same type of Xbox compatibility checks into automated CDRouter test runs. This application note describes the test cases included in the Xbox module along with additional strategies for Xbox compatibility testing.

Xbox Test Cases

CDRouter's xbox.tcl module contains five test cases that are modeled after similar test cases on the Xbox console. Each test case maps directly to one of the tests on the Xbox Live Network Settings: Test XBox Live Connection screen.

IP Address Test

The IP address test verifies that the DHCP client on the LAN can restart DHCP and receive the same IP address. In order to maintain the same port mappings with other end points, it is important that the Xbox console stay registered as the same IP address on the LAN. The test also verifies connectivity to the default gateway and that the DHCP lease is at least five minutes long. If DHCP is not in use, only connectivity to the default gateway is checked.

DNS Test

The DNS test checks that DNS lookups are functioning properly.

MTU Test

To connect to Xbox Live, the router must support an IPv4 MTU of 1364 without fragmenting any packets in the inbound or outbound directions. This test verifies that the device under test (DUT) supports an MTU of 1364.

ICMP Test

The ICMP Test checks that behavior of ICMP Destination Unreachable packets on existing NAT bindings. In order to work with Xbox, a router must not delete a NAT binding if it receives an ICMP Destination Unreachable that matches an existing NAT Binding.

NAT Test

Microsoft classifies NAT implementations as one of three types:

open: The device supports UPnP or Full-Cone NAT.

moderate: The device supports Address-Restricted or Port-Restricted NAT.

strict: The device supports symmetric NAT.

The NAT test case will determine the DUT's NAT type and check for UPnP support. This test case passes if the NAT type is equal to or less string than the target NAT type, which defaults to moderate. The expected NAT type can be configured with the following testvar:

# -- configure the acceptable NAT type: open, moderate, strict
testvar xboxNatMinType moderate

Xbox Live connectivity to other players can be limited depending on the NAT implementation type. More information on these limitations can be found here.

Benefits of Xbox Testing with CDRouter

Since CDRouter controls the entire network in most test setups, there are some additional benefits of running CDRouter's Xbox module in addition to testing with an actual Xbox console. These benefits includes:

ICMP behavior will not be hidden by ISP filtering of ICMP packets
Easily test using all available WAN modes (DHCP, PPPoE, etc.)
Test alongside other common router functionality

Additional Xbox Testing

While running the Xbox module alone will catch functional problems that would prevent Xbox Live Certification, CDRouter also provides additional test modules that can be used to examine in more detail other functionality that is important for Xbox compatibility.

QA Cafe recommends using these other test models when looking at Xbox compatibility:

upnp.tcl The UPnP module verifies UPnP port mappings in much more detail. The Xbox console test for UPnP does not actually create any port mappings. CDRouter's UPnP module does create port mappings and verifies that they work.

nat.tcl The NAT module checks the basic behavior of NAT and helps identify corner cases that could be a problem for the Xbox.

nat-timers.tcl The NAT timers module verifies timeouts for NAT bindings. In order to test the timeouts for NAT bindings, you must know the timeout values based on your implementation. Unexpected short NAT timeouts can cause the Xbox to become disconnected.

And Now on to Xbox Live ...

CDRouter's Xbox module helps catch problems with devices that could lead to Xbox Live compatibility issues. By incorporating this style of testing into CDRouter, the likelihood of encountering Xbox Live compatibility problems is greatly reduced. And this leaves more time for the things we love ... like playing Xbox Live!