# Weixin Mini Program Audio and Video Calling SDK (Linux devices)

This SDK runs on Linux-based hardware devices and provides the device with the ability to make VoIP calls to Weixin Mini Program within WeChat.

Before using this SDK, refer to [Weixin Mini Program Audio and video calling (for hardware]]](../device-voip.md) to complete the pre-built access and Mini Program development process.

# 1. Equipment requirements

# 1.1 Hardware requirements

• CPU: Unspecified minimum requirements, recommended dual-core 500 MHz CPU or higher specifications.
• Memory: 64 M + RAM is recommended. The SDK itself requires at least 13 M of memory.
• disk
  • Space: 32M + ROM (SDK code file itself has 6.5M, in addition to the need to reserve file read and write space).
  • Specifications: EMMC / UFS recommended; If the device contains RPMB, you must ensure that the RPMB partition is not in use.
• microphone
  • Sampling rate: 16000 is required. If not, resampling is required.
  • Encoding format: PCM must be supported; Optional support for OPUS / G711 / G729 (soft coding soft solution).
• Speakers:
  • Sampling rate: 16000 is required. If not, resampling is required.
  • Encoding format: PCM must be supported; Optional support for OPUS / G711 / G729 (soft coding soft solution).
• Camera (optional)
  • Resolution: At least 240 x 320.
  • encoding format
    • H264: Must support, if the device does not support H264, you can choose voice calls.
    • H265: Optional support, only WeChat 8.0.36 and above.
    • Support for other video encoding formats is not considered for now.

# 1.2 System requirements

• C + + Standard Library: Requires libstdc + +.so.6 with minimum version support for C + + 14.
• Time: You need to synchronize the current time, you can't use a fake time
  • Because the SDK needs to establish an SSL link, the machine needs to have a normal world time for TLS to verify the validity of the certificate.
  • The SDK also needs time to determine whether the local cache has expired. If the system does not have the correct time (e.g. 1970-01-01), the SDK will assume that the local cache is consistently not expired, causing the cache to expire without refreshing it.
• Toolchain
  • Arm64: gcc-linaro 7.3.1 and above (glibc 2.17, glibcxx 3.4.22).
  • Arm: gcc-linaro 5.4.1 and above (glibc 2.16, glibcxx 3.4.21).
• Kernel: No minimum requirements

# 1.3 Third party library requirements

The SDK requires the following third-party libraries, which device manufacturers can consider deploying within the system and sharing with other applications to save program size:

1. libcurl: 7.84.0+
2. TLS library: Refer towx_crypto_modulefor details. We strongly recommend that device / chip vendors customize OpenSSL or MbedTLS themselves so that SHA256, RSA, HMAC and other algorithms can use chip hardware acceleration instructions to complete calculations, speed up https certificate verification, and save energy.

libcurl

If libcurl is already included in the system, you can choose between libcurl.so in the SDK and libcurl.so in the system.

• If you use libcurl from the SDK, be sure to replace the libcurl from your system, and don't have two versions at the same time, as there may be compatibility issues.
• The libcurl that comes with the SDK links mbedtls. If you already have OpenSSL or another SSL library on your system, and you don't want to bring an extra copy of mbedtls with your system, you can build your own libcurl version 7.84.0 or above and put it on your system. This eliminates the need for libcurl and mbedtls that come with the SDK.
• The libcurl that comes with the system must support https + TLS certificate client verification, and the version number must be 7.84.0 or above.

# 2. Download the SDK

Download the SDK at here .

After unpacking the SDK, you will see the following directory:

• Include: header files
• Lib: dynamic library file
• Example: Standalone, runnable Demo

The following interfaces are available inincludeis found in theexample]]also has a corresponding example in [[TAG-2- END]]. Please read this document in conjunction with the header file and the example code .

# About the Ringtone

The SDK creates an audio stream to output the ringtone when thewx_voip_session_callfunction is called.The vendor device access party must access the WeChat ring tone , otherwise it will not pass the VoIP access audit.

Ringtones currently provide audio in PCM-S16 (sampling rate 8000). If the device does not support this sampling rate, resample yourself.

# 3. Implementing a hardware abstraction layer

The SDK depends on the hardware and software interface provided by the hardware device platform. Please refer to the Hardware Abstraction Layer documentation to implement the relevant interface.

# 4. Initial SDK

Header file:wmpf / wmpf.h.

After each process starts and before calling the interface of any SDK, you need to call the following method to initialize the SDK.

wx_operation_t wx_init(const wx_init_config_t* config,
                             wx_get_module_t get_module);

Since testing often uses the development or experience version Weixin Mini Program, be careful to set wxa_flavor to the appropriate value when debugging.

parameter

Return value

Wx_operation_t wx_init () is an asynchronous function that initiates network requests.Note the status code after the asynchronous operation is complete.

# 4.1wx_init_config_t

Provides a set of parameters needed to initialize the SDK.

# 5. Register a device

Header file:wmpf / wmpf.h.

After the initial SDK is successful, you should first check whether the device is registered. If you have not registered, you need to register your device before you can initiate a call.

# 5.1 Check whether the current device is registered

wx_error_t wx_device_is_registered(bool* is_registered_out);

parameter

Return value

Wx_error_terror code

# 5.2 Device registration

Register the device with the Weixin Mini Program background.

wx_operation_t wx_device_register(const char* sn_ticket);

NOTE:

1. Once a device is registered, the SDK saves the device's registration information, including the Weixin Mini Program AppID, Model ID, Device ID (SN), and other device registration information that needs to be saved. Therefore, the device can only make Mini Program voice and video calls to specified Mini Programs, cannot change Mini Programs and cannot make Mini Program audio and video calling to multiple Mini Programs.
2. The snTicket is valid for only 5 minutes, and the device cannot store snTickets.
3. The snTicket needs to be passed to the device by the vendor background, rather than the device accessing the WeChat background directly to get the snTicket.Because the snTicket fetch requires accessToken, this parameter needs to be maintained by the vendor background.
4. Although snTicket is only valid for 5 minutes, sn Ticket is only used for verification when the device is registered. As a result, the registration information itself is persistent and sn Tickets are no longer required to register the device or make audio or video calls.

parameter

Return value

Wx_operation_t wx_device_register () is an asynchronous function that initiates network requests.Note the status code after the asynchronous operation is complete.

# 6. Configuring Session

Header file:wmpf / voip.h.

Before initiating a call, you first need to create a session. Session will configure the call initiator (that is, the device access to this SDK) information, set the call status callback, and so on.

# 6.1 Initialize Session object

Callwx_voip_session_new ()to configure thewx_Voip_session_tobject.This object is used to initiate a further call or join a call. Session objects can be reused to initiate or join calls over and over again.

wx_error_t
wx_voip_session_new(wx_voip_session_type_t,
                    wx_voip_session_scene_t,
                    const struct wx_voip_session_listener* listener,
                    void* user_data,
                    const wx_voip_member_t* self_member,
                    const wx_voip_session_config_t* config,
                    wx_voip_session_t* session);

parameter

Return value

Wx_error_terror code

Self_member parameter explaination

Thewx_voip_member_ton the device side (call originator) is set as follows:

# 6.2 Destroy the Session object

When you don't need to initiate a call, you need to destroy the previously created Session object to prevent internal leakage.

void wx_voip_session_destroy(wx_voip_session_t session);

parameter

# 7. call

Header file:wmpf / voip.h.

Thewx_voip_session_type_tobject is successfully created and you can initiate further calls.Callwx_voip_session_call ()to initiate a call, callwx_voip_session_hangup ()voluntarily end the call.The call may also be hung up by the other party, or it may be hung up by an exception, and related events are monitored by the callback set when the session is created.

# 7.1 Start a call

Call asynchronous functionwx_voip_session_call ()to initiate a call:

wx_operation_t
wx_voip_session_call(wx_voip_session_t session,
                     const wx_voip_member_t* callee);

NOTE:

1. Before calling this function to initiate an audio / video call, you need to registerwx_device_is_registeredFunction to determine whether or not the device has been registered (a one-time judgment is sufficient, no more judgments are required).The device can call this function to make Weixin Mini Program audio and video calls only if the device has registered successfully.
2. The OpenID of the WeChat user to dial must come from the Weixin Mini Program and must match the Mini Programs AppID used when the current device was registered.
3. Since the user who is dialing a Weixin Mini Program audio / video call needs to authorize the device in the Mini Program, the device returnsWXERROR_PERMISSION_DENIEDThe user you want to dial should be prompted to do not have permission to make the Mini Program audio and video call.
4. wx_voip_session_can_callFunction determines whether you can make an audio / video call to a specified user (only SDK interface, no background interface).
5. Weixin Mini Program side can determine the authorization of audio / video call bywmpfVoip.getIotContactListfunction.
6. We recommend that vendors maintain authorization correspondence between OpenID and device ID in the background.
7. When debugging, vendors should use the experience or development version Weixin Mini Program because these versions contain audio and video calling experience time by default. And the official Mini Program needs to be approved for audio and video communication before it can go live.

parameter

Return value

Wx_operation_t wx_voip_session_call () is an asynchronous function that initiates network requests.This asynchronous function returns success, which means that the call was correctly dialed and does not mean that the call is connected. Note the status code after the asynchronous operation is complete.

Callee Parameter Dxplaination

Thewx_voip_member_ton the Weixin Mini Program side (call receiver) is set as follows:

# 7.2 Proceed to end the call

wx_operation_t
wx_voip_session_hangup(wx_voip_session_t session,
                       wx_voip_hangup_reason_t reason);

parameter

Return value

Wx_operation_t wx_voip_session_hangup () is an asynchronous function that initiates network requests.This asynchronous function returns success, indicating that the call was successfully hung up. Note the status code after the asynchronous operation is complete.

# 7.3 Mobile Phone WeChat Calling Device - Answering Calls

wx_operation_t wx_voip_listener_join(wx_voip_session_t session,
                                           const char *roomid);

parameter

Return value

Wx_operation_t wx_voip_listener_join () is an asynchronous function that initiates network requests.This asynchronous function returns success, indicating that the call was normally joined. Note the status code after the asynchronous operation is complete.

# 7.4 Cell Phone WeChat Calling Device - Hang up early

This interface is used only before creating a session and is generally used to implement busy lines and disconnect.After creating a session, hangup usingwx_voip_session_hangup.

WX_API wx_operation_t wx_voip_listener_hangup(const char *roomid,
                                              wx_voip_hangup_reason_t reason);

parameter

Introductions This interface does not require a session and can be called directly after wx_init. If a session has been created, you need to use wx_voip_session_hangup.

Return value

Wx_operation_t wx_voip_listener_hangup () is an asynchronous function that initiates network requests.This asynchronous function returns success, indicating that the call was successfully hung up. Note the status code after the asynchronous operation is complete.

# 7.5 Get the call room number roomId

Developers can use roomId to associate both the answering and dialing sides, which is generally used to identify a call and serial reporting information.

const char* wx_voip_session_get_roomid(wx_voip_session_t session);

parameter

Return value

Returns the identity of the room in which the current voip call was made, or roomid. Or return an empty character string.

# 8. Close the SDK

Header file:wmpf / wmpf.h.

When you are no longer using the features provided by the SDK and want to free resources, you can callwx_stop.

Before calling this function, be sure to make sure that everything beforehandwx_operation、wx_voip_sessionWait for the object to be released, otherwise it may have unknown consequences.

After successfully calling this function, if you want to use the SDK again, you need to go through the various processes from the beginning.

wx_error_t wx_stop();

# Return value

Wx_error_terror code

# 9. Common problem

# 9.1 SDK Link Failure

If your device chip is a series such as RV1109, the libstdc + +.so of the toolchain provided by this chip lacks version information, leading to problems finding C + + standard library associated symbols when linking to the SDK we provide.WeChat I don't know what changes this toolchain has made to libstdc + +.so. When this happens, you should seek technical support from the chip manufacturer.For example, replace libstdc + +.so in the toolchain (remember to also replace the system libstdc + +.so).

# 9.2 device/key/get fail: 10008

This means that the SNTicket passed in when you called thewx_device_registerfunction is incorrect or expired.

# 9.3 device/call fail: 9

If this error is reported in the SDK log, it means that the SDK has detected that the WeChat user you are calling does not authorize audio and video dialing access to this device.You need to check the following:

1. Weixin Mini Program When authorizing device audio and video permissions for this WeChat user viawx.requestDeviceVoIP:

• Weixin Mini Program Is the OpenID provided consistent with the ID passed towx_voip_session_callduring device-side calls
• Weixin Mini Program Is the SN used to generate SNTicket the same as the device ID passed in when the device side initializes the SDK. In order to check whether Weixin Mini Program has correctly obtained the authorization for WeChat user's audio and video dialing, the Mini Program developer can call thewx.getDeviceVoIPListThe function obtains a list of authorized devices (device ID / SN) of the current WeChat user to check whether the WeChat user has authorized audio and video rights to a specified device (identified by device ID /SN).

2. Is the device ID passed in at the ID when the device callswx_voip_session_new ()
3. WeChat User unauthorised audio and video access to the device

• WeChat Users delete Weixin Mini Program from WeChat's recent page
• WeChat User unsubscribed in the settings page of Weixin Mini Program In both cases, the WeChat user unlicenses the audio and video of the device.The SDK needs to encounter this error (by determining whether thewx_voip_session_callreturn value isWXERROR_PERMISSION_DENIED), the situation should be handled correctly and the device user should be alerted to the situation.For example, the device user is reminded that "because the WeChat user canceled the authorization of the device, you will not be able to call the WeChat users."

# 9.4 device/call fail: 12

If the error is reported in the SDK log, there are two scenarios:

# Weixin Mini Program Audio and video rights not turned on

If you Weixin Mini Program in the Mini Program background did not open audio and video permissions, you need to open first.

Since Weixin Mini Program official audio and video rights require a device audit to be sent, for device-side developers, init_config - > wa_flavor = WX_WXA_FLAVOR_DEMO for the experience version or WX_WAA_FLAVOR-DEBUG for the Mini Program development version. Consult your Mini Program developer about which version to use.

If you need to use the Weixin Mini Program development version, the device-side developer needs to obtain developer rights for the Mini Program and scan the real-time debugging QR code of the Mini Program-side developer's WeChat developer tool to download the Mini Program's development version.

If you need to use the Weixin Mini Program experience version, then the device side developers need to get the experience of the Mini Program permission, please consult your Mini Program side developers.

# SDK Verification Device Security Failure

First, you need to check if the SDK log contains the following:

CallerTicket gained, you now can create voip session for IoT scene

If the above log is not included, the SDK does verify device security failure.

At this point, you should check to see if the cert file, rpmb_key file, and rpmb_data file exist in the data directory you passed to the SDK. You need to ensure that the data directory has enough disk space to hold these files, and these files need to be persistent, can not be overwritten or deleted during the firmware upgrade, can not be tampered with. If these files are tampered with, you may encounter another error before you encounter OperateWxData fail: 12.

# 9.5 device/call fail: 13

If this error is reported in the SDK log, it means that you have changed the device ID after the device has been initialized with the SDK once.

Note that the device ID must have a one-to-one mapping to the device, and the device ID on a device should not be changed. If you encounter this error, you should use the Device ID of the original Device Initialization SDK instead of replacing the device ID.

# 9.6 device/call fail: 20

The incoming OpenID is incorrect and is not a valid OpenID.Make sure that the incoming OpenID is obtained by Weixin Mini Program wx.login.

# 9.7 device/register: HTTP 0 XX

This means that the SDK encountered an HTTPS error when registering the device, where XX should be a number, such as device / register: HTTP 077,This represents an error code 77. Please check the error code table in C URL to see what the error code corresponds to an error. If you can fix the error yourself, such as a network connection failure, please do so yourself.

Ensure that the system network environment can access the following domains: servicewechat.com, ae.weixin.qq.com

# 9.8 Device / register: errcode: 9800011 errmsg: sn_ticke Illegal

This indicates that the incoming SNTicket is incorrect.

SNTicket format is a combination of numbers and letters, 32 characters long, and contains no other symbols.Be careful not to mispass other parameters, such as access_token. Also, to avoid security risks from devices or Weixin Mini Program acquiring accestoken, you should provide an interface for generating sntickets to Mini Programs and devices at the end of the service.

# 9.9 device/register: errcode: 9 errmsg: device `xxx`not registered

This means that the given device ID is already tied to a device. Note that the device ID must have a one-to-one mapping with the device, and the device ID cannot be shared by multiple devices and cannot be migrated from one device to another. A device is also not allowed to change the device ID. If you do the above, such as continuing to use the registered device ID on another device, you should switch to a different device ID on the "other device."

# 9.10 [CheckMemoryLeak]: Detected memory leak: wx_operation, leak objects: 1

This error log indicates that you did not process the return value when calling the wx_voip_xxx family of functions, or other functions that return the wx_operation object. When calling a function that returns the wx_operation object, you must call wx_operation_await, wx_operations_wait, or wx_action_destroy for its return value to either wait asynchronously for the operation to complete, synchronously for the action to complete, or not to wait for the operation completed to release the wx-operation object.

# 9.11 Found local timestamp incorrect, expected XXX, actual XXX

This error indicates that your system's timestamp is incorrect. Make sure that the difference between the device system's time stamp and current time is not more than 5 seconds.

The SDK obtains the system time stamp through thetime (NULL)function. Since the time stamp does not have a time zone, make sure that the time stamp and UTC time match is obtained this way.

# 9.12 Weixin Mini Program The camera image collected by the display device is rotated 90 degrees / The picture is stretched

Weixin Mini Program VoIP plug-in provides setUIConfig The interface allows manufacturers to solve the problem of 90 degrees of rotation (camera Rotation) and the picture stretching deformation (aspectRatio).

if (/* 微信版本 <= 8.0.40 */) {
  wmpfVoip.setUIConfig({
    // 设置视频通话时 caller 画面
    callerUI: {
      cameraRotation: 0,  // caller的视频画面旋转角度，有效值为 0, 90, 180, 270。默认 0
      aspectRatio: 4/3, // 纵横比，caller的视频画面会进行适配比例，有效值 数字。默认 4/3
      horMirror: false, // 横向镜像，boolean 值，默认 false
      vertMirror: false, // 竖直镜像，同上
      enableToggleCamera: true, // 是否支持切换摄像头，false 则不显示「摄像头开关」按钮。默认false 【该配置项在wmpf无效，wmpf默认开摄像头，且不显示开关按钮】
    },
  })
}

# 9.13 [wx_operation_validate]: invalid address: wx_operation 0xXXXXXXXXX

That means you're callingwx_operation_wait、wx_operation_await、wx_operation_destroyThewx_operationobject passed in while waiting for function is incorrect, or the object has been released. Check the code for correctness.

# 9.14 Call wx_camera_stream_listener::data with stream 0xXXXXXXXX, user_data 0xXXXXXXXX. But it's released.

You called the wx_camera_stream_listener:: data function at the wrong time. Note that wx_camera_streamopen is not to start calling the listener:: data function; The listener:: data function cannot be called after wx_camera_streamclose. Thread security also needs to be taken into account. If you maintain wx_camera_stream through a list, then when traversing the list, you need to lock with open and close to avoid data competition.