Nabto WebRTC Error And State Handling

To make a robust WebRTC application, it is necessary to implement robust error handling and state handling. Errors can occur in the system which can either be reported into a central Application Performance Management system or be handled locally in the application itself.

State changes occur when the system goes through a series of expected states, sometimes the system ends in an unexpected state or certain actions need to be taken to get out of a state. This document goes into the specific details about Error and State handling.

Error Handling

In a Nabto WebRTC application there are several sources for errors, which a robust application needs to handle.

A common Nabto WebRTC client or device application consists of the following components which can emit errors:

  • Nabto WebRTC Signaling (Client or Device)
  • Device only: Nabto WebRTC Signaling Channels
  • Nabto WebRTC Signaling (Client or Device) Message Transport(s)
  • WebRTC Peer Connection(s) (from the chosen WebRTC implementation)

A Nabto WebRTC Signaling Client can generally go into a FAILED state where a Nabto WebRTC Signaling Device will generally try to avoid switching to a FAILED state at all cost.

This distinction comes from the fact that a Nabto WebRTC Device application often is an application which is designed to run for a long period of time. Often the application is started when the device starts and is expected to be running as long as the device is powered on. For this reason error handling in a Nabto WebRTC device differs from error handling in a Nabto WebRTC client.

A client can end in a failed state where it has given up after some time, a device will generally not end in a failed state because it keeps trying to stay connected to the Nabto WebRTC Signaling Service even if it means that it has to keep retrying forever. A client will stop retrying after a fixed number of retries. In this case the application must restart the connection, this is often initiated by the user doing some action to actively restart the WebRTC connection.

Nabto WebRTC Signaling Service Errors

The Nabto WebRTC Signaling Client or Device can emit errors if errors occur when communicating with the Nabto WebRTC Signaling Service. These errors can be general network errors or errors from the Nabto WebRTC Signaling Service. A list of error which comes from the Nabto WebRTC Signaling Service can be found at Http Errors

An example of how to listen for errors in the Nabto WebRTC Signaling Client can be found in Signaling Client Error Handling

An example of how to listen for errors in a Nabto WebRTC Signaling Device can be found in Signaling Device Error Handling

Nabto WebRTC Signaling Errors

Signaling errors occur on the signaling channel between a client and a device (camera). They are sent from a client towards a device or from a device to a client. Each error consists of an errorCode and an optional errorMessage describing the error.

The protocol can be extended with custom application specific error codes. If that happens these custom error codes need to start with an APPLICATION_ to distinguish them from the error codes which are defined in the Nabto WebRTC Signaling protocol.

In a Nabto WebRTC Signaling Client application tbe Nabto WebRTC Signaling Errors comes from either the clients on(“error”) listener or from the associated Client Message Transport. See Client Error Example for how it is implemented in a Nabto WebRTC Client Application.

In a Nabto WebRTC Device Applicationn the Nabto WebRTC Signaling Errors occurs on either the Nabto WebRTC Signaling Channel or the associated Device Message Transport. The Device Error Handling Example shows how to listen for these errors in a Nabto WebRTC Device Application.

State Handling

In a Nabto WebRTC application can listen for state changes in several components:

  • The Nabto WebRTC Signaling Client changes Nabto WebRTC Signaling Connection States
  • The Nabto WebRTC Signaling Client changes Nabto WebRTC Signaling Channel States
  • The WebRTC Connection (RTCPeerConnection) changes WebRTC Connection States

Signaling Connection State

The Nabto Signaling Connection State describes the state of the connection between the Nabto WebRTC Signaling Client or the Nabto WebRTC Signaling Device and the Nabto WebRTC Signaling Service. Generally if the client or device is in the CONNECTED state everything is fine. It is also completely normal that a Client or Device briefly is in a WAIT_RETRY or CONNECTING state. This can happen if the application is initially connecting or a network change occurs, such as the user is changing the network from WiFi to LTE, etc.

See Connection States for a list of possible states.

See Client State Handling for how to handle connection state changes in a client.

See Device State Handling for how to handle connection state changes in a device.

Signaling Channel State

The Nabto WebRTC Signaling Channel State describes the state of the signaling channel between the Nabto WebRTC Signaling Client and the Nabto WebRTC Signaling Device.

The CONNECTED state means that the SDK considers the signaling channel to the remote peer to be active and functioning. This assessment is based on the current channel status and recent activity.

The state may remain CONNECTED briefly even if the remote peer has disconnected unexpectedly, e.g. if a peer disconnects ungracefully.

If the state is DISCONNECTED, the SDK has detected that the other peer is no longer connected to the signaling service and signaling messages cannot be delivered.

This state can be used to indicate to the application whether the remote peer appears to be online or offline.

WebRTC Peer Connection State

The WebRTC Connection State reflects the state of the connection between the peers as managed by the WebRTC implementation. There are a few other related states, such as signalingState and iceConnectionState, which are not covered in this document. For details on these, refer to the official documentation of the WebRTC implementation in use. For instance for browser based applications, see MDN.

The connection state of a RTCPeerConnection tells the application whether a WebRTC Connection is connected to the remote peer or not. When this state is connected everything is generally fine. Beside using this state to inform the user application about if everything is fine or not, this state can be used to initiate calls to checkAlive() in the Nabto WebRTC Signaling Client or the Nabto WebRTC Signaling Channel, and it can be used to initiate RTCPeerConnection iceRestart()

See WebRTC Connection States for a list of common WebRTC Connection States found in WebRTC libraries.

The following Example show how to listen for WebRTC connection state changes in a WebRTC application.

Client Examples

Client example showing how to handle errors

Below is an example client snippet which only focuses on error handling:

JavaScript Swift Java
// Somewhere in MyClass
do {
    signalingClient = createSignalingClient(signalingClientOptions)
    await signalingClient.addObserver(self)
    try await signalingClient.start()
    messageTransport = try await createClientMessageTransport(
        client: signalingClient,
        options: .sharedSecret(sharedSecret: sharedSecret)
    )
    await messageTransport?.addObserver(self)
} catch (error) {
    // Handle thrown errors
}

// Extension of MyClass to implement Observer interfaces.
extension MyClass: MessageTransportObserver, SignalingClientObserver, RTCPeerConnectionDelegate {
    func messageTransport(_ transport: any MessageTransport, didFinishSetup iceServers: [SignalingIceServer]) async {
        peerConnection = factory.peerConnection(with: config, constraints: constraints, delegate: self)
    }

    func peerConnection(_ peerConnection: RTCPeerConnection, didChangeConnectionState newState: RTCPeerConnectionState) {
        if newState == .failed {
            // An error occurred on the RTC peer connection.
        }
    }

    func messageTransport(_ transport: any MessageTransport, didError error: any Error) async {
        if let error = error as? SignalingError {
            // A signaling error occured on the signaling channel between the
            // Client and the Device.
        } else {
            // The error comes from a component outside the Nabto WebRTC SDK.
        }
    }

    func signalingClient(_ client: SignalingClient, didError error: Error) async {
        if let error = error as? HttpError {
            switch error {
                case .productIdNotFound(let statusCode, let message):
                    // The product id does not exists in the Nabto WebRTC Signaling service.
                    break
                case .deviceIdNotFound(let statusCode, let message):
                    // The device id does not exist in the Nabto WebRTC Signaling service.
                    break
                case .unknown(let statusCode, let message):
                    // The nabto webrtc signaling service returned an error for the request.
                    break
            }
        } else if let error = error as? SignalingError {
            // A signaling error occurred on the signaling channel between the
            // Client and the Device.
        } else if let error = error as? DeviceOfflineError {
            // the device is not online but was required to be online.
        } else {
            // The error comes from a component outside the Nabto WebRTC SDK.
        }
    }
    // ... other Observer and Delegate functions
}


var client = SignalingClientFactory.createSignalingClient(signalingClientOptions);
var messageTransport = ClientMessageTransport.createSharedSecretMessageTransport(client, ...);
var peerConnection;
messageTransport.addObserver(new MessageTransport.AbstractObserver() {
    @Override
    public void onError(Throwable error) {
        super.onError(error);
        if (error instanceof SingnalingError) {
            // A signaling error occured on the signaling channel between the
            // Client and the Device.
        } else {
            // The error comes from a component outside the Nabto WebRTC SDK.
        }
    }
    @Override
    public void onSetupDone(List<SignalingIceServer> iceServers) {
        super.onSetupDone(iceServers);
        ...
        peerConnection = peerConnectionFactory.createPeerConnection(..., new PeerConnection.Observer() {
            @Override public void onConnectionChange(PeerConnection.PeerConnectionState newState) {
                super.onConnectionChange(newState);
                if (newState == PeerConnection.PeerConnectionState.FAILED) {
                    // An error occurred on the RTC peer connection.
                }
            }
        }
    }
});

client.addObserver(new SignalingClient.AbstractObserver() {
    @Override
    public void onError(Throwable error) {
        super.onError(error);
        if (error instanceof ProductIdNotFoundException) {
            // The product id does not exists in the Nabto WebRTC Signaling service.
        } else if (error instanceof DeviceIdNotFoundException) {
            // The device id does not exist in the Nabto WebRTC Signaling service.
        } else if (error instanceof HttpException) {
            // The nabto webrtc signaling service returned an error for the request.
        } else if (error instanceof SingnalingError) {
            // A signaling error occurred on the signaling channel between the
            // Client and the Device.
        } else if (error instanceof DeviceOfflineException) {
            // the device is not online but was required to be online.
        } else {
            // The error comes from a component outside the Nabto WebRTC SDK.
        }
    }
});


const client = createSignalingClient(...);
client.on("error", (error /*: unknown*/) => {
  if (error instanceof HttpError) {
    // The error originates from the Nabto WebRTC signaling Service.
  } else if (error instanceof SignalingError) {
    // Signaling errors relate to the signaling between the client and the device.
  } else {
    // Another error such as a system error e.g. related to underlying network problems or programming mistakes in dynamic typed languages.
  }
})
const messageTransport = createClientMessageTransport(...)
messageTransport.on("error", (error /*: unknown*/) => {
  if (error instanceof SignalingError) {
    // A signaling error occurred, the error has already been sent to the other peer.
    // e.g. VERIFICATION_ERROR or DECODE_ERROR.
  } else {
    // Another error occurred, that could e.g. be a system error or an error indicating an error in the code in dynamic typed languages.
  }
})
messageTransport.on("setupdone", (iceServers /*: RTCIceServers[]*/) => {
  const peerConnection = new RTCPeerConnection(...)
  peerConnection.addEventListener("connectionstatechange", () => {
    if (peerConnection.connectionState === "failed") {
      // An error occured on the RTC peer connection.
    }
  })
})

Client example showing how to handle state changes

Below is a code snippet which listens on state changes:

JavaScript Swift Java
// Somewhere in MyClass
do {
    signalingClient = createSignalingClient(signalingClientOptions)
    await signalingClient.addObserver(self)
    try await signalingClient.start()
    messageTransport = try await createClientMessageTransport(
        client: signalingClient,
        options: .sharedSecret(sharedSecret: sharedSecret)
    )
    await messageTransport?.addObserver(self)
} catch (error) {
    // Handle thrown errors
}

// Extension of MyClass to implement Observer interfaces.
extension MyClass: MessageTransportObserver, SignalingClientObserver, RTCPeerConnectionDelegate {
    func messageTransport(_ transport: any MessageTransport, didFinishSetup iceServers: [SignalingIceServer]) async {
        peerConnection = factory.peerConnection(with: config, constraints: constraints, delegate: self)
    }

    func peerConnection(_ peerConnection: RTCPeerConnection, didChangeConnectionState newState: RTCPeerConnectionState) {
        if newState == .failed {
            // An error occurred on the RTC peer connection.
        }
    }

    public func signalingClient(_ client: any SignalingClient, didConnectionStateChange connectionState: SignalingConnectionState) async {
        if connectionState == .connected {
            // The client is connected to the Nabto WebRTC Signaling Service.
        }
    }

    public func signalingClient(_ client: any SignalingClient, didChannelStateChange channelState: SignalingChannelState) async {
        // Handle the Signaling Channel State.
    }
    // ... other Observer and Delegate functions
}



var client = SignalingClientFactory.createSignalingClient(signalingClientOptions);
var messageTransport = ClientMessageTransport.createSharedSecretMessageTransport(client, ...);
var peerConnection;
messageTransport.addObserver(new MessageTransport.AbstractObserver() {
    @Override
    public void onSetupDone(List<SignalingIceServer> iceServers) {
        super.onSetupDone(iceServers);
        ...
        peerConnection = peerConnectionFactory.createPeerConnection(..., new PeerConnection.Observer() {
            @Override public void onConnectionChange(PeerConnection.PeerConnectionState newState) {
                super.onConnectionChange(newState);
                if (newState == PeerConnection.PeerConnectionState.FAILED) {
                    // An error occurred on the RTC peer connection.
                }
            }
        }
    }
});

client.addObserver(new SignalingClient.AbstractObserver() {
    @Override
    public void onConnectionStateChange(SignalingConnectionState state) {
        super.onConnectionStateChange(state);
        if (state == SignalingConnectionState.CONNECTED) {
            // The client is connected to the Nabto WebRTC Signaling Service.
        }
    }
    @Override
    public void onChannelStateChange(SignalingChannelState state) {
        super.onChannelStateChange(state);
        // Handle the Signaling Channel State.
    }
});


const client = createSignalingClient(...);
client.on("connectionstatechange", (state /*: SignalingConnectionState*/) => {
  if (state === SignalingConnectionState.CONNECTED) {
    // The client is connected to the signaling service.
  }
})
client.on('channelstatechange', (state /*: SignalingChannelState*/) => {

})

const messageTransport = createClientMessageTransport(...)
messageTransport.on("setupdone", (iceServers /*: RTCIceServers[]*/) => {
  const peerConnection = new RTCPeerConnection(...)
  peerConnection.addEventListener("connectionstatechange", () => {
    if (peerConnection.connectionState === "failed") {
      // An error occured on the RTC peer connection.
    }
  })
})

Nabto WebRTC Signaling Device Examples

Example Device State Change listener

This example shows how to listen for Device Connection State Changes, the state which says if the device is connected to the Nabto WebRTC Signaling Service or not.

JavaScript C++
const device = createSignalingDevice(...)
device.on("connectionstatechange", (state /*: SignalingConnectionState*/) => {
})


auto device = nabto::webrtc::SignalingDeviceFactory::create(...);
device->addStateChangeListener([](nabto::webrtc::SignalingDeviceState state){
})

Example Device Signaling Channel State Changes

This example shows how to listen for Signaling Channel State changes in a device.

JavaScript C++
const device = createSignalingDevice(...)
device.onNewSignalingChannel = async (channel /*: SignalingChannel*/, authorized /*: boolean*/) => {
  channel.on("channelstatechange", (state /*: SignalingChannelState*/) {
    // handle the state change
  })
}


auto device = nabto::webrtc::SignalingDeviceFactory::create(...);
device->addNewChannelListener([](
      nabto::webrtc::SignalingChannelPtr channel,
      bool authorized)
    {
      channel->addStateChangeListener([](nabto::webrtc::SignalingChannelState state){
        // handle the state change
      })
    });

Signaling Channel Error handling

This example shows how to listen for errors on a Nabto WebRTC Signaling Channel:

JavaScript C++
const device = createSignalingDevice(...)
device.onNewSignalingChannel = async (channel /*: SignalingChannel*/, authorized/*: boolean*/) => {
  channel.on("error", (error/*: SignalingError*/) {
    console.log(`A Signaling Error occurred error code: (${error.errorCode}), error message: (${error.errorMessage}))`);
  })
}


auto device = nabto::webrtc::SignalingDeviceFactory::create(...);
device->addNewChannelListener([](
      nabto::webrtc::SignalingChannelPtr channel,
      bool authorized)
    {
      channel->addErrorListener([](const SignalingError& error){
        std::cout << "A Signaling Error occurred error code: ("
                  << error.errorCode() << "), error message: ("
                  << error.errorMessage() << ")" << std::endl;
       })
    })

Signaling Device Sending Protocol Defined Error

This example shows how to send an error which is predefined in the Nabto WebRTC Signaling Protocol.

JavaScript C++
const device = createSignalingDevice(...)
device.onNewSignalingChannel = async (channel /*: SignalingChannel*/, authorized /*: boolean*/) => {
  // example for sending a predefined error code.
  channel.sendError(
    new SignalingError(
      SignalingErrorCodes.ACCESS_DENIED,
      "Access to the device was denied"));
}


auto device = nabto::webrtc::SignalingDeviceFactory::create(...);
device->addNewChannelListener([](
      nabto::webrtc::SignalingChannelPtr channel,
      bool authorized)
    {
      channel->sendError(
        new nabto::webrtc::SignalingError(
          nabto::webrtc::SignalingErrorCode.ACCESS_DENIED,
          "Access to the device was denied"));
    })

Signaling Device sending Application defined Error

This example shows how to send a custom application defined error.

JavaScript C++
const device = createSignalingDevice(...)
device.onNewSignalingChannel = async (channel /*: SignalingChannel*/, authorized /*: boolean*/) => {
  // Example on sending
  channel.sendError(
    new SignalingError(
      "APPLICATION_CUSTOM_ERROR_CODE",
      "This is a custom error sent from the device."));
}


auto device = nabto::webrtc::SignalingDeviceFactory::create(...);
device->addNewChannelListener([](
      nabto::webrtc::SignalingChannelPtr channel,
      bool authorized)
    {
      channel->sendError(
        new nabto::webrtc::SignalingError(
          "APPLICATION_CUSTOM_ERROR_CODE",
          "This is a custom error sent from the device."));
    })

Appendix

HTTP Errors

HTTP Errors occur between a Nabto WebRTC Signaling Client and the Nabto WebRTC Signaling Service. Each HTTP Error consists of a HTTP Status Code such as 404 and a message describing the error such as e.g. The specified product id does not exist

A few of the Http Errors are specified by a subclass describing a specific error condition. These specific error conditions are ProductIdNotFoundError and DeviceIdNotFoundError.

see Nabto WebRTC Signaling protocol for a list of defined http error conditions.

Signaling Errors

Signaling errors occur on the signaling channel between a client and a device (camera). They are sent from a client towards a device or from a device to a client. Each error consists of an errorCode and an optional errorMessage describing the error.

The protocol can be extended with custom application specific error codes. If that happens these custom error codes need to start with an APPLICATION_ to distinguish them from the error codes which are defined in the Nabto WebRTC Signaling protocol.

The following error codes are defined in the Nabto WebRTC Signaling Protocol.

export enum SignalingErrorCodes {
  /**
   * The SDK received a message with invalid JSON or a JSON object not following the protocol
   */
  DECODE_ERROR = "DECODE_ERROR",
  /**
   * A signed message was received, but the signature could not be verified.
   */
  VERIFICATION_ERROR = "VERIFICATION_ERROR",
  /**
   * This error is received from the other peer when it is closing the Signaling Channel.
   */
  CHANNEL_CLOSED = "CHANNEL_CLOSED",
  /**
   * This error is sent to the SignalingClient by the SignalingDevice if the client is using an unknown Channel ID.
   */
  CHANNEL_NOT_FOUND = "CHANNEL_NOT_FOUND",
  /**
   * This can be sent by the device if a client attempts to create a new channel but the device has reached its limit.
   */
  NO_MORE_CHANNELS = "NO_MORE_CHANNELS",
  /**
   * This error code is sent if a channel is rejected based on authentication and/or authorization data.
   */
  ACCESS_DENIED = "ACCESS_DENIED",
  /**
   * This error is sent if the device or the client encounters an internal error.
   */
  INTERNAL_ERROR = "INTERNAL_ERROR"
}

Nabto WebRTC Signaling Connection State

The Nabto WebRTC Signaling Connection State is an enum describing the state of the interaction between the Nabto WebRTC Signaling Client or the Nabto WebRTC Signaling Device and the Nabto WebRTC Signaling Service.

The state changes when the state of the connection between the client or device and the Nabto WebRTC Signaling Service changes.

The following Signaling Connection States exist:

export enum SignalingConnectionState {
  /**
   * The Signaling Connection was just created
   */
  NEW = "NEW",
  /**
   * The Signaling Connection is connecting to the backend.
   */
  CONNECTING = "CONNECTING",
  /**
   * The Signaling Connection is connected and ready to use.
   */
  CONNECTED = "CONNECTED",
  /**
   * The Signaling Connection is disconnected and waiting for its backoff before reconnecting.
   */
  WAIT_RETRY = "WAIT_RETRY",
  /**
   * The Signaling Connection has failed and will not reconnect.
   */
  FAILED = "FAILED",
  /**
   * The Signaling Connection was closed by the application.
   */
  CLOSED = "CLOSED"
}

The Nabto WebRTC Signaling Connection State can transition between the following states:

NEW

FAILED

CONNECTING

CONNECTED

CLOSED

WAIT_RETRY

Nabto WebRTC Signaling Channel State

The Nabto WebRTC Signaling Channel State describes the state of the signaling channel between the Nabto WebRTC Signaling Client and the Nabto WebRTC Signaling Device.

The CONNECTED state means that the SDK considers the signaling channel to the remote peer to be active and functioning. This assessment is based on the current connection status and recent activity.

The state may remain CONNECTED briefly even if the remote peer has disconnected unexpectedly, e.g. if a peer disconnects ungracefully.

If the state is DISCONNECTED, the SDK has detected that the other peer is no longer connected to the signaling service and signaling messages cannot be delivered.

This state can be used to indicate to the application whether the remote peer appears to be online or offline.

The Nabto WebRTC Signaling Channel State can have the following states:

export enum SignalingChannelState {
  /**
   * The state is new if the client has not been connected yet.
   */
  NEW = "NEW",
  /**
   * The state is connected if our best guess is that the other peer is currently connected to the signaling service.
   */
  CONNECTED = "CONNECTED",
  /**
   * The state is DISCONNECTED if the current best guess is that the other peer is disconnected from the signaling service.
   */
  DISCONNECTED = "DISCONNECTED",
  /**
   * If the channel has received an error, which is fatal in the protocol, the state will be failed.
   */
  FAILED = "FAILED",
  /**
   * If close has been called on the channel by the application or by the signaling client, the state is closed.
   */
  CLOSED = "CLOSED"
}

The Nabto WebRTC Signaling Channel State can transition between the following states:

NEW

CONNECTED

DISCONNECTED

FAILED

CLOSED

WebRTC PeerConnection State

The WebRTC Connection State is the state of the WebRTC/DTLS connection between the peers, this is a state which resides in the WebRTC implementation used by the application. See the implementation’s documentation for details, e.g. see MDN for browser based applications.

The WebRTC Connection State has the following states: new, connecting, connected, disconnected, failed or closed.

new The initial state of the WebRTC connection.

connecting This means that the WebRTC connection is in the progress of creating a connection, ie. exchanging SDP descriptions and ice candidates and using this information to establish DTLS and SRTP connections.

connected This state means that there are a WebRTC connection between the peers and that the best guess is that it is working properly.

connected This state means that the WebRTC implementation considers the peer-to-peer connection to be established and functioning correctly. This assessment is based on the success of ICE and DTLS negotiations and ongoing liveness checks. The state may remain connected for a short time even if the connection has silently failed, e.g. if a peer disconnects ungracefully. This state can be used to indicate that media and data channels are expected to function, although actual media flow is not guaranteed.

disconnected Often a keepalive mechanism runs on the DTLS part of the WebRTC Connection. If the keep alive mechanism detects a problem with the connection the state does to disconnected. For browser based implementations this is often around 5-10 seconds after a problem occurs. In this state is is a good idea to probe the system for errors, if the default SignalingEventHandler is used a checkAlive() is invoked on the Nabto WebRTC Signaling Connection.

failed If the WebRTC Connection has been determined to be non functioning. If the default SignalingEventHandler is used an iceRestart is issued in this state to try to recover the connection. This state often occurs after around 15 seconds of failing DTLS keepalive probes.