RTSP Video Streaming using Nabto Edge
Video streaming from IP cameras, DVRs and NVRs is a very popular Nabto Edge use case. In most cases, Nabto Edge WebRTC is the modern and preferred approach. But the classical way of using Nabto with cameras through Nabto Edge TCP Tunnels may be the best choice for some scenarios: It requires less resources than WebRTC and is extremely simple to integrate.
Nabto Edge TCP Tunnels are used with an existing RTSP service: By injecting a pair of Nabto Edge TCP tunnels as seen in the following diagram, integration is extremely simple:
In the RTSP video scenario, the “existing TCP server application” is the RTSP server running on the camera (or gateway or DVR/NVR). The “existing client TCP application” is an RTSP video player component like VLC or FFmpeg.
The player is simply pointed towards the TCP proxy server running inside Nabto Edge Client SDK, e.g. rtsp://127.0.0.1:51743/live/0 and the remote feed is shown - through a secure connection.
Note: The Nabto TCP server running locally on the client cannot distinguish TCP connections from your existing application and any other applications running on the localhost. If the host running the client cannot be trusted, application level authentication on top of your TCP connection is needed (e.g., RTSP or HTTP basic auth). Or integration through low level Nabto Edge Streams can be used instead (much more complex integration).
This vs other video related Nabto Edge guides
This guide describes the API interaction necessary to support video streaming through tunnels. 5 other guides exist that are relevant for video streaming:
The Tunnel Video guides for iOS and Android walk through installing and using full production ready player apps (full source available), including a Docker hosted RTSP service for testing.
The general TCP Tunnel step-by-step guide describes how to configure a general commandline tunnel client and setting up the reverse proxy on the camera. This is good for desktop testing through a standalone player (e.g., VLC) and is often used when developing the camera side of the solution.
The Raspberry Pi video example walks through how to setup a Nabto Edge Tunnel on a Raspberry Pi to stream video.
The Linux IP Camera integration guide describes in detail how to cross compile for a typical Linux-based IP camera and to install and use the resulting binary.
The guidelines in this document also apply to any other TCP use case, e.g. if you need to tunnel HTTP or a custom TCP based protocol instead of RTSP as described in this document.
Opening a TCP tunnel to an RTSP Server
The following example shows how you can open a TCP tunnel to an RTSP server on a camera. This tunnel is opened on a connection that is established as described in the Connecting guide.
class RtspExample {
var tunnel: Tunnel?
func openTunnelAndPlay(connection: Connection) throws {
self.tunnel = connection.createTcpTunnel()
self.tunnel.open(service: "rtsp", localPort: 0) // note 1
let port = try self.tunnel.getLocalPort() // note 2
RtspPlayer.instance.openUrl("rtsp://127.0.0.1:\(port)/live")
}
func stop() throws {
self.tunnel?.close()
}
}
public class RtspExample {
TcpTunnel tunnel;
// RtspPlayer is your video player of choice, on Android this would commonly be ExoPlayer.
RtspPlayer player;
public void openTunnelAndPlay(Connection connection) {
tunnel = connection.createTcpTunnel();
tunnel.open("rtsp", 0);
int port = tunnel.getLocalPort();
player.openUrl("rtsp://127.0.0.1:" + port + "/video");
}
public void stop() {
if (tunnel != null) {
tunnel.tunnelClose();
tunnel.close();
}
tunnel = null
}
}
class RtspExample {
var tunnel: TcpTunnel?
// RtspPlayer is your video player of choice, on Android this would commonly be ExoPlayer.
val player: RtspPlayer
fun openTunnelAndPlay() {
tunnel = connection.createTcpTunnel()
tunnel.open("rtsp", 0)
val port = tunnel.getLocalPort()
player.openUrl("rtsp://127.0.0.1:${port}/video")
}
fun stop() {
tunnel?.let { it.close() }
tunnel = null
}
}
using Nabto.Edge.Client;
public class RtspExample
{
private ITcpTunnel? _tunnel;
// Replace this RtspPlayer with your video player library of choice, such as VLC.
private RtspPlayer _player = new RtspPlayer();
public async void OpenTunnelAndPlay(IConnection connection)
{
_tunnel = connection.CreateTcpTunnel();
await _tunnel.OpenAsync("rtsp", 0);
var port = _tunnel.GetLocalPort();
_player.OpenUri($"rtsp://127.0.0.1:{port}/video");
}
public async void Stop()
{
if (_tunnel != null)
{
await _tunnel.CloseAsync();
}
_tunnel = null;
}
}
As seen, it is extremely simple to open a tunnel to use with RTSP: Once the connection is established, it is just a few lines of code needed to start streaming remotely.
Two details need a bit further explanation:
Specifying what to connect to on the camera
In the tunnel open step (note 1 in the snippet), a service name and a local port number is specified. The service name must match what is configured on the device side. This defines which TCP service that the Nabto Edge TCP Tunnel application running on the device (camera) connects to. Setting this up is described in the Tunnel Video example and in the commandline client step-by-step guide.
In the example, rtsp is specified in the client (at note 1), meaning that this service is requested from the device: The device tunnel application looks up in its configuration which local TCP service should be connected to, for instance the camera’s tunnel configuration could look as follows:
$ cat ~/.nabto/edge/config/tcp_tunnel_device_services.json
[
{
"Host" : "127.0.0.1",
"Id" : "rtsp",
"Port" : 554,
"Type" : "video"
}
]
So in this example, when the client requests the rtsp service, the Nabto Edge TCP Tunnel device application connects to TCP port 554 on 127.0.0.1 on the camera (where the device application is running).
Specifying what to listen on inside the client
When opening a TCP tunnel, the Nabto Edge Client SDK starts a TCP server inside the client. It listens on a port - this is what is denoted as localPort at note 1.
When the user does not see the local port number, it is recommended to just listen on port 0 as seen in the example. This means using an ephemeral port, ie the operating system allocates an available port. The actual port allocated can the be retrieved in the next step (note 2). This port where the tunnel is listening is what is passed on to the player component.
It is important to understand that the local port the Nabto Edge Client SDK listens on (as specified in note 1) is completely unrelated to the TCP port of the target service! If for instance the RTSP service on the camera listens on port 554, you should most likely not specify port 554 as localPort when opening the tunnel! Which port to connect to on the device is configured through the service name as described in “Specifying what to connect to on the camera”.
In a mobile app, the user does not have to see this local port number and an ephemeral (random) port is fine. If the user actually needs to see the TCP tunnel port, a specific port can be used - for instance if starting a TCP tunnel command line client application to use the tunnel from e.g. a desktop video player. Then with the static port, you don’t have to update the client application with a new port number at each invocation.
Troubleshooting
Despite the simplicity of integrating tunnels in your client application, errors can still occur due to the many moving parts involved. The following describes the most common errors encountered and how to remedy.
Note that the interpretation of the below error codes are specific for opening tunnels: The same error codes are used in other contexts where the reason and remedy is different than listed here.
Always handle all errors
In general, to understand why a behavior different than expected is observed, it is necessary to follow standard programming good practices and carefully handle all errors: If for instance pairing fails in an early step, the pairing error should not be ignored but handled there without proceeding.
If just bluntly continuing on such error by suppressing exceptions or ignoring error codes, confusing errors occur later, for instance when opening a tunnel (where you will see error FORBIDDEN). And if ignoring that too, the TCP client will fail with an even less specific error - the RTSP player will complain about not being able to connect to the local proxy endpoint.
“Forbidden” (error code FORBIDDEN)
When an exception or error FORBIDDEN (native SDK error code NABTO_CLIENT_EC_FORBIDDEN) is observed when opening a tunnel, it means that the current user is not allowed to open a tunnel to the specific service.
If using the default configuration of the standard tunnel device application, this most likely means the current user is not successfully paired. See the Connecting guide to learn how to setup pairing.
You can manually inspect the IAM state on the device (camera) side if you believe pairing should have been completed:
$ cat ~/.nabto/edge/state/tcp_tunnel_device_iam_state.json
{
"FriendlyName" : "Tcp Tunnel",
"InitialPairingUsername" : "admin",
"LocalInitialPairing" : false,
"LocalOpenPairing" : false,
"OpenPairingPassword" : "zSlAbuWJxlr",
"OpenPairingRole" : "Standard",
"OpenPairingSct" : "PX0UASP9dAs",
"PasswordInvitePairing" : true,
"PasswordOpenPairing" : true,
"Users" : [
{
"Password" : "aC4ixKRteNEa",
"Role" : "Administrator",
"ServerConnectToken" : "tbjYXHwxAexJ",
"Username" : "admin"
},
{
"Fingerprint" : "572d6912d4200f0e1736bd90eb2a40a8c763c371da3c2ec53aa5f9fb93bc0710",
"Role" : "Standard",
"ServerConnectToken" : "A3q5y3CBEExQ",
"Username" : "jdoe"
}
]
}
In the users section, you can see the list of users on the device. A paired user has a fingerprint set. So in the above example, only the user jdoe has been paired.
If you do not see the expected user in the list, this is the reason for the FORBIDDEN error. So repeat the pairing step and carefully observe which error occurs.
If pairing was indeed successfully completed and you see the expected user in the list, the assigned role is not allowed to open a tunnel if you see the FORBIDDEN error. With the default role definitions, the Standard or Administrator roles are required to be able to open a tunnel. That is, if you see Guest as role in the configuration file, the user will not be able to open a tunnel. To learn about permissions and roles in general, read the Nabto Edge IAM guide.
Solution: To summarize the above, you must make sure pairing succeeds and that the paired role has permissions to open a tunnel.
“Not found” (error code NOT_FOUND)
When an exception or error NOT_FOUND (native SDK error code NABTO_CLIENT_EC_NOT_FOUND) is observed when opening a tunnel, it means that a service is requested that is not configured on the device. In the previous example, if for instance the service http is requested, the error is seen as only rtsp is configured in the camera’s config.
Solution: Make sure the client (mobile app) and the device (camera) agree on the service: Either add the service on the camera (and restart the tunnel process) or use another service in the client.
Permission denied when opening a tunnel
Solution: Check that you are not specifying a privileged port, ie a port number lower than 1024. Also see the recommendation above about using ephemeral ports whenever possible (specified as local port 0).
Empty reply
If the connection is immediately closed by the Nabto Client SDK’s local proxy server when your TCP client connects, it likely means that the target TCP service on the device (camera) is not reachable.
You can confirm if this is the case by inspecting the output of Nabto Edge TCP tunnel device application when you try to connect with your TCP client to the opened tunnel in the client:
######## Configured TCP Services ########
# Id Type Host Port
# rtsp video 127.0.0.1 554
########
Attached to the basestation
_tcp_tunnel_connection.c:148 ERROR - Could not connect to tcp endpoint: 127.0.0.1:554
_tcp_tunnel_connection.c:148 ERROR - Could not connect to tcp endpoint: 127.0.0.1:554
The TCP tunnel is opened successfully in this scenario, the Nabto Edge connection is fine and the user has been assigned an appropriate role. Your client application is also able to connect to the local TCP proxy server in the client. But the Nabto Edge TCP tunnel application on the device is not able to connect to the target TCP server.
You can confirm if this possible by trying to connect to the same port by hand when logged into to the target device (the camera). Just try to connect to the port using some simple available utility like netcat or telnet:
$ telnet 127.0.0.1 554
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
If you are not able to connect when logged in to the camera, the Nabto Edge TCP tunnel application will not be able to either. Make sure both of the following is ok:
Solution 1: Make sure the target TCP service on the device is running on the port as configured in the device’s service config:
$ cat ~/.nabto/edge/config/tcp_tunnel_device_services.json
[
{
"Host" : "127.0.0.1",
"Id" : "rtsp",
"Port" : 554,
"Type" : "video"
}
]
And confirm with telnet or netcat on the camera as suggested above that this configuration is indeed valid.
Solution 2: Make sure the loopback interface (lo) is up and that the address is as expected:
Often on embedded Linux systems, the loopback interface (lo) is not up automatically. Check with ifconfig and observe the lo interface is configured. If not, you need to configure it - it is out of scope of this document, but it normally looks something along the lines of:
$ ifconfig lo 127.0.0.1 up
This must then be done automatically at boot.
The loopback interface does not necessarily have to be bound to the 127.0.0.1 address, the entire 127.0.0.0/8 block is actually valid loopback addresses. Indeed, we have seen systems where lo has been assigned something different than 127.0.0.1. Change your tunnel config accordingly.
Long delay in player before RTSP feed starts
The default in most player components is to use RTSP over UDP. The Nabto Tunnels are TCP based. So if you do not configure the player to use RTSP over TCP, the player will have to time out on its UDP attempts before it falls back to TCP.
Solution: All players and player components have an option to always connect using RTSP/TCP. Set this option when tunnelling RTSP using Nabto TCP Tunnels.