TR-369 Training - An overview of the User Services Platform (USP/TR-369)

Watch our full webinar overview of USP with our special guests from ARRIS

Note - this webinar was given in September of 2018. There have been many advancements and development of resources for USP implementations (such as the addition of a Controller REST API and the addition of MQTT as a transport). Read below for more.

USP/TR-369 - The User Services Platform

USP is a revolutionary standard that builds on the lessons learned from TR-069. In addition to being a faster and more flexible at meeting the existing use cases of TR-069, USP is built to enable:

  • Managed Wi-Fi - USP is ideal for building the managed Wi-Fi systems that are in high demand by today’s service providers, with flexible transport mechanisms, a multi-controller architecture, and standardized Wi-Fi management objects.
  • Bulk statistics collection - USP lets parties develop applications that gather the telemetry necessary to mass data processing, AI, and machine learning.
  • Standardized IoT onboarding - USP unifies the fragmented world of IoT and smart home devices by proxying non-IP protocols like ZigBee, Zwave, etc.; onboarding and managing device lifecycles; and standardizing IoT interfaces and service objects in the Device:2 data model (TR-181).
  • Customer self-care applications - USP Controllers can be built for end-user devices to allow for customer self-care without dependence on cloud resources.
  • Application enabled smart gateways - USP extends and enhances the software module management features of TR-069 to allow “dockerized” device management and the lifecycle management of third-party applications like security, Wi-Fi telemetry, and parental controls.
  • Secure communication - All USP applications can benefit from end-to-end application-layer TLS connections specified in the standard and robust trust establishment and access control for controllers.

A deep exploration of these use cases can be found in the Broadband Forum white paper Realizing the Promise of the Connected Home with USP.

Resources for development

Standards and data model resources

The specification for architecture, discovery, end-to-end message encoding, transport, and types, plus security and access control are defined in Broadband Forum TR-369 User Services Platform (USP). This can be found on the web at https://usp.technology.

The data model for describing the service elements exposed by USP Agents are defined in the Device:2 Root Data Model (published as TR-181 Issue 2). The models for CWMP and USP pull from the same common core with some minor changes for protocol-specific management objects. These models can be found at https://usp-data-models.broadband-forum.org.

Open source USP Agent

The Broadband Forum maintains an open source USP Agent on which developers can base their implementations. The Agent is improved through frequent releases. It is a great starting point for implementers while building a complete solution and testing it with CDRouter.

It can be found on GitHub here: https://github.com/BroadbandForum/obuspa

Testing and validating your implementation

The CDRouter USP add-on contains test cases to validate your USP implementation, acts as a scriptable USP Controller, and is frequently updated with new test cases produced by the Broadband Forum. In addition, it has exhaustive validation for your implementation of the Device:2 data model. QA Cafe was involved in the development of USP early on and has built a solid base of expertise to assist you in your own development.

Learn more…

Overall architecture

USP consists of a network of Controllers and Agents that manipulate “Service Elements”. Service Elements are one or more objects defined in the Agent’s supported data model, and exposed as the Agent’s instantiated data model. A Controller may be an Autoconfiguration Server (ACS) similar to TR-069, or controlled by an application service provider to control specific elements, or even a Controller on a smartphone or in the gateway to manipulate elements in the user’s network.

Protocol stack

USP Protocol Stack

USP Records

All USP messages are encapsulated in a USP record. These records are used to guarantee integrity of the USP messages and provide a layer of security when security can’t be accomplished at the message transport layer. USP Records allow an optional Session Context that allows for protected messages (payloads) and the segmentation and re-assembly of USP messages when large messages need to traverse intermediate proxies.

Message transport agnostic

TR-069 is bound heavily to HTTP. In the design of USP, there was a desire to ensure that the protocol be cleanly layered so that transport of USP messages could be accomplished over multiple “transports” (the term Message Transfer Protocol was adopted to differentiate between message transport and the actual OSI transport layer). Message Transfer Protocols (MTPs) defined for USP 1.1 include the COnstrained Application Protocl (CoAP), the Simple Text-Oriented Messaging Protocol, Message Queuing Telemetry Transport and WebSockets over HTTP.

Protocol buffers encoding

Rather than sending XML documents over HTTP, USP makes use of Protocol Buffers to encode messages in transport. Protocol Buffers encoding is binary, and ends up saving significant space. It also has advantages over JSON as the encoded fields are defined in one or more schemas (.proto files) that each endpoint can use to encode or decode messages reliably and interoperably. In USP, there is a schema for the USP Record (usp-record.proto) whose payload field contains a USP Message (usp-msg.proto).

For example, this is the Get message from the usp-msg.proto schema:

message Get {
  repeated string param_paths = 1;
}

message GetResp {
  repeated RequestedPathResult req_path_results = 1;

  message RequestedPathResult {
    string requested_path = 1;
    fixed32 err_code = 2;
    string err_msg = 3;
    repeated ResolvedPathResult resolved_path_results = 4;
  }

  message ResolvedPathResult {
    string resolved_path = 1;
    map<string, string> result_params = 2;
  }
}

Evolution from TR-069

Backwards compatibility with Device:2

One of the development goals of USP was to serve as a natural evolution of TR-069, allowing TR-069 users to seamlessly migrate to USP without having to re-architect their OSS/BSS or device/driver integration. USP makes use of the Device:2 data model with some modifications to allow the definition of commands and events with the data model.

Reduced, flexible message set

USP attempts to handle both RESTful and non-RESTful situations with a reduced message set that is made extensible by moving many operations into the Device:2 data model. As such, it includes CRUD (Create, Read, Update, Delete) based operations, plus a Notify message for notifications to be sent from the Agent to a Controller, and a Operate message to call functions (commands) defined in the Agent’s supported data model. These messages include:

  • Add - used to create new instances of multi-instance objects in the Agent’s instantiated data model.
  • Set - used to alter the state of an object’s parameters in the Agent’s instantiated data model.
  • Delete - used to remove instances of objects in the Agent’s instantiated data model.
  • Get - used to retrieve the value of one or more parameters of objects in the Agent’s instantiated data model.
  • GetInstances - used to retrieve the instances and instance identifiers of objects that can have multiple instances in the Agent’s instantiated data model.
  • GetSupportedDM - used to retrieve information about the Agent’s supported data model, including the supported objects and their parameters, commands, and events.
  • GetSupportedProtocol - used to retrieve the USP versions the Agent supports. In version 1.0, the only valid value is 1.0.
  • Notify - used by the Agent to alert a Controller of events that were triggered based on one or more subscriptions that a Controller configures in the Agent. These include some standardized messages such as ValueChange, ObjectCreation, ObjectDeletion, OnBoardRequest, and OperationComplete, plus a generic Event notification to carry information about object specific events defined in the Agent’s supported data model. Events that were defined in CWMP but are now defined in the Device:2 data model include things like BOOT, TRANSFER_COMPELTE, etc.
  • Operate - used to call functions (called commands) defined by objects in the Agent’s supported data model, including a way to provide arguments to those commands, and retrieve return arguments. The mechanism includes support for both synchronous and asynchronous operations; the former returns arguments as part of the OperateResponse from the Agent, and the latter may return arguments in an associated OperationComplete event. Examples of operations that were RPCs in CWMP, but commands in USP, include things like Reboot(), Download() (which operates on Device.Firmware instances), etc.

Lighter weight messages

We mentioned above that Protocol Buffers saves space on the wire by creating standardized fields that are binary encoded in transport. This reduces complexity significantly.

In addition, USP messages that use the Device:2 data model (defined originally for CWMP) makes use of several features that reduce the amount of object path information that needs to be sent by the Controller and returned by the Agent:

  • Relative paths - USP uses the concept of “relative paths” when addressing objects or parameters in the Agent’s database. For example, let’s say a Controller made a Get request on the Device.Wifi.SSID.1. object, in order to retrieve all of the parameters of the SSID. In CWMP, this would require the CWMP Endpoint to return:
Device.Wifi.SSID.1.Enable = <value>
Device.Wifi.SSID.1.Status = <value>
Device.Wifi.SSID.1.Alias = <value>
Device.Wifi.SSID.1.Name = <value>
Device.Wifi.SSID.1.LastChange = <value>
Device.Wifi.SSID.1.LowerLayers = <value>
Device.Wifi.SSID.1.BSSID = <value>
Device.Wifi.SSID.1.MACAddress = <value>
Device.Wifi.SSID.1.SSID = <value>
Device.Wifi.SSID.1.Upstream = <value>

And more, for every sub-object of the SSID object.

In USP, the Agent only needs to return:

Device.Wifi.SSID.1.
  Enable = <value>
  Status = <value>
  Alias = <value>
  Name = <value>
  LastChange = <value>
  LowerLayers = <value>
  BSSID = <value>
  MACAddress = <value>
  SSID = <value>
  Upstream = <value>

Etc., for the remaining parameters, sub-objects, and their parameters in the Agent’s database.

  • Addressing by key - Secondly, USP provides new ways of addressing instances of objects. Rather than try to wrestle with instance numbers, USP allows the addressing of an object instance by the object’s unique key. For example, the Device.NAT.PortMapping table has a unique key made up of three parameters; RemoteHost, ExternalPort, and Protocol. A particular PortMapping could be addressed as:

Device.NAT.PortMapping.[RemoteHost==""&&ExternalPort==0&&Protocol=="TCP"]

If the instance number was not known or to keep consistency with multiple managed devices.

  • Search Expressions - Lastly, USP provides a complex yet powerful means of searching through a set of objects for only those objects who have parameters that match certain criteria. For example, you could have the Agent return IPv4 Addresses for all IP Interfaces with a Normal type and Static addressing type that have at least 1 Error Sent with the following search expression:

Device.IP.Interface.[Type=="Normal"&&Stats.ErrorsSent>0].IPv4Address.[AddressingType=="Static"].IPAddress

Search expressions can be much more complex than this, of course, and include different operators and other complex syntaxes defined in the specification.

Optional session context with fewer round-trips

In USP, sessions are optional and used for security and integrity purposes. Even when sessions are used, communication channels are “nailed up” so that there’s no need to establish a USP session for every message. Moreover, there is no need for Connection Requests to initiate communication sessions, since Controllers can send messages (which results in session initiation, if desired) directly to an Agent at any time.

Our own developer Tucker made a hilarious graphic that contrasts the “noisiness” of TR-069 versus USP:

TR-069 vs. USP conversation length

Multi-Controller architecture

Lastly, USP opens the doors to a vastly more flexible ecosystem by exploding the one-to-one relationship that existed between a CWMP Endpoint and an ACS in TR-069. A given USP Agent can be manipulated by multiple Controllers, which may be in a Service Provider’s operational network (like today’s ACS), a Controller operated by a 3rd party application vendor, a Controller on a gateway facing the LAN, or even on an end-user’s smartphone.

USP accomplishes this by assigning access control rights to each Controller. This is configurable in the data model using the Device.LocalAgent.ControllerTrust. object, which allows the definition of Roles that may or may not apply to different objects and parameters in the Agent’s database.

For example, a Service Provider may have access rights to upgrade the firmware on a device that they have deployed in the end-user’s network. When giving monitoring control to the end-user, who wants to see a report of connected hosts, the Agent may limit a Controller on the end-user’s smartphone so it can’t accidentally mess with the firmware or other critical configuration aspects.

Build your USP/TR-369 solution

Test against an automated, scriptable USP Controller with thousands of pre-defined test cases.

Learn more

Develop your solution even faster with comprehensive automated test cases and a scriptable USP Controller.