DVBAPI
======
DVB API stands for Linux DVB Application Programming Interface, so in short it is a set of API calls which are used on
linux to handle DVB hardware. From the OSCam point of view the most interesting part is to be able to provide all data
necessary for channel decryption. The OSCam DVBAPI module was written to handle this work.

Architecture
============
A DVBAPI module needs the following information to decrypt a channel:
- PMT table (information from TV receiver software about the requested channel for livetv/recording and the ECM PIDs)
- CAT table (needed to get information about EMM type and PIDs)
- Filtered ECM/EMM data

If OSCam is able to decrypt a service, information about the decrypted PIDs (audio, video, etc) and CW (keys)
is sent back to the TV receiver software from the CAM device.

History
=======
The first and "standard" use case is probably Enigma. OSCam creates a /tmp/camd.socket. Enigma sends the PMT data to
this socket and as a result OSCam opens the necessary DVB demux devices (e.g. /dev/dvb/adapter0/demux0) and filters
for ECM, CAT and EMM data. These data are then parsed by the OSCam dvbapi module and as a result the CA_SET_PID and
CA_SET_DESCR ioctl calls are made, leading to proper decryption. All this was working on the same hardware and the
same DVB devices were used. This kind of usage was mainly for linux STB.

Next step was generic PC support, by extending the dvbapi module to send PIDs and keys back to TV software (initially
via a special UDP socket, later via the same /tmp/camd.socket). The TV software was able to use this information in
software decryption (DeCSA).

At some point, the OpenPLi team created a new CaPMT interface, which was then implemented in OSCam (as pmt_mode=6).
It is described here: http://wiki.openpli.org/caPMT
The main feature was reverting the roles: OSCam now acts as a client and connects to /tmp/.listen.camd.socket created
by Enigma. This way multiple Software CAMs could be running and connecting to Enigma's .listen.camd.socket. Another
important improvement in this mode (also implemented in OSCam) was the ability to handle extra CA_PMT list managements.
This allows to use one socket connection to handle more than one channel at a time (previously clients had to manage
a single connection to /tmp/camd.socket per subscribed channel).

As the .listen.camd.socket mode makes less sense on generic PC platform (the OSCam is still server, while the client
could be any PC software used), the second feature which allows handling multiple channels on single socket connection
was extended to cover other modes (not only pmt_mode=6) in OSCam.

Network mode
============
The last feature that was added was a network mode. The change was made to be able to connect to an OSCam instance
which is not running on the same machine where the TV receiver software (and a DVB hardware) runs.

Why not use dedicated protocols like newcamd/camd in such cases?
- To have ECM/EMM handling in OSCam, where it belongs. It is better maintained and fixes come in quicker.
- OSCam knows what readers it has, so it could do load balance/filtering/priorities etc.

As a result, instead of /tmp/camd.socket (which could be used only on the same machine) a listening_socket parameter
was added. So the unix domain socket switched to a fully-featured TCP socket which can be connected from any network
client.

As a result besides CA_SET_PID and CA_SET_DESCR new calls were passed to socket: DMX_SET_FILTER and DMX_STOP. The TV
receiver software has to filter the demux itself (according to the new calls above) and send results like ECM/EMM/CAT
data back to OSCam using the same connection. Because OSCam was only aware of PMT data on the socket, a new
DVBAPI_FILTER_DATA command (0xffff0000) was added to handle client data from filters.

This way, communication between the TV receiver software and OSCam could be finally done using only one single TCP
connection. Moreover, the demux is only accessed by a single TV receiver software process, which from the architecture's
point of view is definitely a better solution.

New protocol description (socket commands)
===========================================
As there are more and more dvbapi clients, some problems start to appear. First of all there was some kind of mess
because OSCam's network mode doesn't take into account the endianness in first form of the network protocol. Second,
it was not consistant (e.g. PID was always send as little endian in DMX_STOP, while the rest depend on OSCam's host
architecture). Finally the standard API ioctl codes for CA_SET_PID, CA_SET_DESCR, DMX_SET_FILTER and DMX_STOP behave
differently. These codes are composed by a macro which takes into account the length of the associated structures and
on some hardware the first bits of the MSB was different. So the clients had to do some strange workarounds with
the MSB byte: fix 0x80 -> 0x40 and 0x20 -> 0x00 when needed

Finally, the first byte sent to client was an adapter index, which was not always needed in all commands. Now the
first 4-byte integer is unique operation code, so a client will know what is the request type and could read the rest
of data according to the following description (and field sizes).

To address all above problems and additionally make smooth transitions when updating the protocol in the future there
was added some kind of "handshake" for clients. All new implementations should use it. Currently the old and new
implementations should work fine, but in the future a network client which will not introduce itself (thus not providing
it's supported protocol version) may be dropped/ignored by OSCam.

All multibyte integers (if not specified otherwise) should be send using network byte order, so your client should use
ntoh() functions when receiving. OSCam is doing hton() before sending and vice versa.

Just right after a client connects to an OSCam network socket, it should send a greeting in format:

-= DVBAPI_CLIENT_INFO =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_CLIENT_INFO
uint16_t                 protocol version supported by client
uint8_t                  size of followed string (255 bytes max)
string                   name and version of the client (string length should be max 255 bytes)

The server will respond with a similar reply:

-= DVBAPI_SERVER_INFO =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_SERVER_INFO
uint16_t                 protocol version supported by OSCam
uint8_t                  size of followed string (255 bytes max)
string                   OSCam version and build (string length should be max 255 bytes)

Next, when a client wants to start a channel, it should send the PMT data (program map table). The PMT data structure
starts with constant AOT_CA_PMT (0x9F8032). The data format of the CA_PMT is described in chapter 8.4.3.4 (page 30) of
the EN 50221 PDF (european standard).

-------------------------------------------------------------------------------

Please note that OSCam is expecting a number of private descriptors injected
into the CA PMT data. They shall be located inside the program info loop. All
supported descriptors follow the structure defined in 2.6 of Rec. ITU H.222.0
and they are the following:

---------------------------------------------------------------------
| descriptor_tag |           Identification           |    Usage    |
---------------------------------------------------------------------
|      0x81      |  enigma_namespace_descriptor       |  optional   |
|      0x82      |  demux_ca_mask_device_descriptor   |  deprecated |
|      0x83      |  adapter_device_descriptor         |  mandatory  |
|      0x84      |  pmt_pid_descriptor                |  mandatory  |
|      0x85      |  service_type_mask_descriptor      |  optional   |
|      0x86      |  demux_device_descriptor           |  mandatory  |
|      0x87      |  ca_device_descriptor              |  mandatory  |
---------------------------------------------------------------------

Descriptors marked as "mandatory" shall be present in the CA PMT message, in
order OSCam to get all necessary information for the specified program. Below
is a detailed description for each of the supported descriptors with its
structure and usage.

-------------------------------------------------------------------------------

1. Enigma namespace descriptor

Provides additional information for the program specified, such as enigma
namespace (orbital position, frequency and polarization), transport stream id
and original network id. Although its presense in the CA PMT is optional, it
is advised to be included as OSCam utilizes these information in many aspects.

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| enigma_namespace_descriptor(){       |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     enigma_namespace                 |      32       |   uimsbf   |
|     transport_stream_id              |      16       |   uimsbf   |
|     original_network_id              |      16       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x81.

descriptor_length - The length of the descriptor is equal to 0x08.

ens - This is the enigma namespace as defined in OpenPLi, openATV and other
open enigma2 images. An example implementation can be found at:
https://github.com/OpenPLi/enigma2/blob/develop/lib/dvb/frontend.cpp#L476

transport_stream_id – The transport stream id as defined in various DVB SI
tables.

original_network_id - The original network id as defined in various DVB SI
tables.

-------------------------------------------------------------------------------

2. Demux and ca mask device descriptor

It was used to provide information about the demux device as well as the ca
device(s) carrying the specified program. It was created for the DM7025 set top
box and it is now considered deprecated. Many hosts use this descriptor in a
different way (carrying different information) than it was originaly designed
leading to confusion. OSCam will continue to support this descriptor as some
legacy hosts still use it. For newer hosts, the adapter_device_descriptor (tag
0x83), the demux_device_descriptor (tag 0x86) and the ca_device_descriptor (tag
0x87) shall be used for sending the necessary data to OSCam.

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| demux_ca_mask_device_descriptor(){   |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     ca_mask                          |       8       |   uimsbf   |
|     demux_device                     |       8       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x82.

descriptor_length - The length of the descriptor is equal to 0x02.

ca_mask - It is a bit mask of the ca device(s) carrying the specified program.
Bit 0 corresonds to ca0, bit 1 corresponds to ca1 and so on.

demux_device - The demux device that carries the specified program. It is
limited to values between 0x00 and 0x08 in older enigma2 images.

-------------------------------------------------------------------------------

3. Adapter device descriptor

Provides information about the adapter device carrying the specified program.

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| adapter_device_descriptor(){         |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     adapter_device                   |       8       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x83.

descriptor_length - The length of the descriptor is equal to 0x01.

adapter_device - The adapter device that carries the specified program. It can
take values from 0x00 to 0xFF, thus a maximum number of different 256 adapters
are supported.

-------------------------------------------------------------------------------

4. PMT pid descriptor

Provides information about the PMT pid of the specified program.

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| pmt_pid_descriptor(){                |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     pmt_pid                          |      16       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x84.

descriptor_length - The length of the descriptor is equal to 0x02.

pmt_pid - The pid that carries the PMT table of the specified program.

-------------------------------------------------------------------------------

5. Service type mask descriptor

It provides information about the type (live tv, recording, streaming service,
or any combination) of the program specified. It's currently not unitilized in
OSCam and its usage in considered optional.

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| service_type_mask_descriptor(){      |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     service_type_mask                |      32       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x85.

descriptor_length - The length of the descriptor is equal to 0x04.

service_type_mask - This is a bit mask of the different service types enabled
for the specified program. Service type values are defined in enigma2 and can
be found at https://github.com/OpenPLi/enigma2/blob/develop/lib/dvb/pmt.h#L135

-------------------------------------------------------------------------------

6. Demux device descriptor

It is used to provide information about the demux device carrying the specified
program. It is a replacement to the deprecated demux_ca_mask_device_descriptor
(tag 0x82), as it supports up to 256 different demux devices.

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| demux_device_descriptor(){           |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     demux_device                     |       8       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x86.

descriptor_length - The length of the descriptor is equal to 0x01.

demux_device - The demux device that carries the specified program. Its value
can be between 0x00 and 0xFF.

-------------------------------------------------------------------------------

7. Ca device descriptor

This descriptor provides OSCam with information about the ca device carrying
the specified program. It is created as a replacement to the deprecated
demux_ca_mask_device_descriptor (tag 0x82). It has the following syntax:

---------------------------------------------------------------------
|                Syntax                |  No. of bits  |  Mnemonic  |
---------------------------------------------------------------------
| ca_device_descriptor(){              |               |            |
|     descriptor_tag                   |       8       |   uimsbf   |
|     descriptor_length                |       8       |   uimsbf   |
|     ca_device                        |       8       |   uimsbf   |
| }                                    |               |            |
---------------------------------------------------------------------

decriptor_tag - The tag of the descriptor is equal to 0x87.

descriptor_length - The length of the descriptor is equal to 0x01.

ca_device - The ca device that carries the specified program. Its value can be
between 0x00 and 0xFF (256 different ca devices supported).

-------------------------------------------------------------------------------

After OSCam parses the PMT data, it starts filtering ECM PIDs. It sends the following request to the client:

-= DVBAPI_DMX_SET_FILTER =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_DMX_SET_FILTER
uint8_t                  adapter index
uint8_t                  demux index
uint8_t                  filter number
*** The following data are the fields from the dmx_sct_filter_params structure (added separately to avoid padding problems)
uint16_t                 pid
uint8_t[16]              filter data (filter.filter)
uint8_t[16]              filter mask (filter.mask)
uint8_t[16]              filter mode (filter.mode)
uint32_t                 timeout
uint32_t                 flags

The client should then filter the data and pass it back to OSCam using the following frame:

-= DVBAPI_FILTER_DATA =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_FILTER_DATA
uint8_t                  demux index
uint8_t                  filter number
uint8_t[]                filtered data from demux

When OSCam is able to decrypt a channel, it initially sends a list of PIDs associated with the descrambler index using
this packet:

-= DVBAPI_CA_SET_PID =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_CA_SET_PID
uint8_t                  adapter index
ca_pid_t                 8-byte ca_pid_t structure (the pid and index fields are in network byte order)

And also sends the CW for decryption:

-= DVBAPI_CA_SET_DESCR =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_CA_SET_DESCR
uint8_t                  adapter index
ca_descr_t               16-byte ca_descr_t structure (the index and parity fields are in network byte order)

When OSCam wants to inform the client about stopping a filter, it sends the following packet:

-= DVBAPI_DMX_STOP =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_DMX_STOP
uint8_t                  adapter index
uint8_t                  demux index
uint8_t                  filter number
uint16_t                 PID to stop filtering

When the client closes connection, all associated channels are stopped in OSCam.

Alternatively when there is a need to stop channel decoding, while having the connection still open you can send a
special '3f' packed to OSCam. To stop decoding the specified demux, the following CA_PMT data should be sent to OSCam:
9F 80 3f 04 83 02 00 <demux index>
If <demux index> is 0xff, then it is parsed as a wildcard and all demuxers associated with the connection are stopped.

In protocol version 2 the new packet with ECM info data was introduced:

-= DVBAPI_ECM_INFO =-
-----------------------------------------------------------------------
type/size                description
-----------------------------------------------------------------------
uint32_t                 operation code -> DVBAPI_ECM_INFO
uint8_t                  adapter index
uint16_t                 Service ID
uint16_t                 CAID
uint16_t                 PID
uint32_t                 Provider ID
uint32_t                 ECM time (ms)
uint8_t                  size of followed string (255 bytes max)
string                   cardsystem name (string length should be max 255 bytes)
uint8_t                  size of followed string (255 bytes max)
string                   reader name (string length should be max 255 bytes)
uint8_t                  size of followed string (255 bytes max)
string                   from - source name (string length should be max 255 bytes)
uint8_t                  size of followed string (255 bytes max)
string                   protocol name (string length should be max 255 bytes)
uint8_t                  hops (cccam & gbox; set to 0 otherwise)
