# Mini Program audio and video call SDK (Linux equipment)

this SDK Running on board. Linux On the hardware device of the system, provide the device with a call to the Mini Program within the mobile phone WeChat VoIP Ability to talk.

Use this SDK Before, please refer to theMini Program audio and video calls (for Hardware) >Complete the front access and Mini Program development process.

# 1. Equipment requirements

# 1.1 Hardware requirements

• CPU: Undetermined minimum requirements, dual-core recommended 500MHz CPU Or higher.
• Memory: Recommended 64M+ RAM。SDK Itself at least needs to be 13M Memory.
• disk
  • Space: 32 M + ROM (SDK The code file itself has 6.5M, In addition, you need to reserve file read and write space)。
  • Specifications: Recommended EMMC/UFSIf the device contains RPMB, then you must ensure that RPMB The partition is not used.
• microphone
  • Sampling rate: support required 16000, which is not supported, requires the vendor to resample itself.
  • Encoding Format: Must be supported PCMOptional support OPUS/G711/G729 (Soft Coding and Soft Solution).
• Speakers:
  • Sampling rate: support required 16000, which is not supported, requires the vendor to resample itself.
  • Encoding Format: Must be supported PCMOptional support OPUS/G711/G729 (Soft Coding and Soft Solution).
• Camera (optional)
  • Resolution: At least 240x320。
  • Encoding format
    • H264: Must be supported, If the device does not support H264, You can choose voice calls.
    • H265: Optional Support, WeChat only 8.0.36 And above versions are supported.
    • Support for other video encoding formats is not considered.

# 1.2 System requirements

• C++ Standard library: Need to come with libstdc++.so.6, Version Minimum Support C++14。
• The time: Need to synchronize the current time, can not use false time
  • Because SDK Need to establish SSL Link, the unit needs to have normal world time in order to TLS Verify certificate validity.
  • SDK It also takes time to determine whether the local cache has expired. If the system does not have the correct time (Such as 1970-01-01), be SDK Will think that the local cache consistently does not expire, The cache was not refreshed after causing the cache to time out.
• Tool chain
  • arm64: gcc-linear 7.3.1 And above version (glibc 2.17, glibcxx 3.4.22).
  • arm: gcc-linear 5.4.1 And above version (glibc 2.16, glibcxx 3.4.21).
• Kernel: Unspecified minimum requirements

# 1.3 Third-party library requirements

SDK The following third-party libraries are required that device manufacturers can consider deploying within the system and sharing with other applications to save program volume:

1. libcurl: 7.84.0+
2. TLS Library: Please refer to wx_crypto_Module Chapters. We strongly recommend that equipment/Chip manufacturers customize their own OpenSSL or MbedTLS, so that it can be used in the SHA256、RSA、HMAC Such algorithms can use the chip's hardware acceleration instructions to complete the calculation and speed up the https Certificate verification speed while saving energy.

**libcurl **

If the system already contains libcurl, You can find it at SDK Inner libcurl.so And the system that comes with libcurl.so Choose one or the other. There is no need to carry multiple libcurl。

• If used SDK Inner Libcurl, be sure to replace the Libcurl, don't have two versions at the same time. There may be compatibility issues.
• SDK Comes with the inside libcurl Linked. mbedtls。If you already have it in the system OpenSSL Or other SSL Library, and you do not want to carry an extra copy in the system Mbedtls, you can build your own 7.84.0 Or above version libcurl And placed in the system. So there's no need SDK Bring your own libcurl and mbedtls And...
• Comes with the system. libcurl Must support https+TLS Certificate client validation, the version number needs to be 7.84.0 Or above version.

# 2. to download SDK

Please contact us at this place to download SDK。

SDK After unpacking, you will see the following table of contents:

• include: Header file
• lib: Dynamic library file
• example: Independently operable Demo case

The following describes the interfaces, all of which can be found in include Found in the directory,example There is also a corresponding example in the. Read this document in conjunction with the header file and sample code。

# About Ringtones

SDK Will be in the process of calling wx_voip_session_call Function to create an audio stream to output the ringtone. The manufacturer's equipment access party must be connectedWeChat Ring Tones, otherwise it will not be possible to pass VoIP Access audit.

Ringtones are currently provided by PCM-S16 (sampling rate 8000) Audio in the format. If the device does not support this sampling rate, resample yourself.

# 3. Implement the hardware abstraction layer

SDK The normal operation depends on the hardware and software interfaces provided by the hardware device platform, please refer toHardware abstraction layerDocument implements the relevant interface.

# 4. to initialize SDK

Header file:wmpf/wmpf.h。

After each process starts, call any SDK You need to call the following method on the SDK Initialize.

wx_operation_t wx_init(const wx_init_config_t* config,
                             wx_get_Module_t get_Module)

Since testing often uses the development version or experience version of the Mini Program, please pay attention to the debugging when setting wxa_flavor To the appropriate value.

parameter

Return value

wx_operation_t wx_init()Is an asynchronous function that initiates a network request. Note the status code after the asynchronous operation completes.

# 4.1 wx_init_config_t

Provides initialization SDK The set of parameters required.

# 5. Registration device

Header file:wmpf/wmpf.h。

SDK After successful initialization, you should first check whether the device has been registered. If you are not registered, you need to register your device before you can initiate a call.

# 5.1 Check if the current device is registered

wx_error_t wx_device_is_registered(bool* is_registered_out)

parameter

Return value

wx_error_t Error code

# 5.2 Equipment registration

Register your device with WeChat Mini Program.

wx_operation_t wx_device_register(const char* sn_ticket)

Notes:

1. After the device is registered, the SDK Will save device registration information, including Mini Programs AppID, Model ID, equipment ID (SN), and other device registration information that needs to be saved. Therefore, the device can only make Mini Program audio and video calls to the designated Mini Program, cannot change the Mini Program, and cannot make Mini Program audio and video calls to multiple Mini programs.
2. SnTicket only 5 Minute validity period, the device may not store snTicket.
3. SnTicket Need to be passed to the device by the manufacturer's background, rather than the device directly accessing the WeChat background to get snTicket。because SnTicket Access to Needs Access Token, which needs to be maintained by the vendor background.
4. although SnTicket only 5 Minutes of validity, but SnTicket Used only for verification at device registration. Therefore, the registration information itself is persistent, and follow-up is no longer required. SnTicket Register your device or make audio and video calls.

parameter

Return value

wx_operation_t wx_device_register()Is an asynchronous function that initiates a network request. Note the status code after the asynchronous operation completes.

# 6. To configure Session

Header file:wmpf/voip.h。

Before initiating a call, you need to first create a Session。Session The call initiator (that is, access to this SDK Of the device) information, set the call state callback, etc.

# 6.1 to initialize Session object

call wx_voip_session_new() To configure wx_voip_session_t Object. This object is used to initiate or join a call further. Session Objects can be reused to initiate or join calls over and over again.

wx_error_t
wx_voip_session_new(wx_voip_session_type_t,
                    wx_voip_session_scene_t,
                    const struct wx_voip_session_listener* listener,
                    void* user_data,
                    const wx_voip_member_t* self_member,
                    const wx_voip_session_config_t* config,
                    wx_voip_session_t* session)

parameter

Return value

wx_error_t Error code

self_member Dxplaination of parameters

The device side (the call originator) of the wx_voip_member_t The settings are as follows:

# 6.2 Destroy Session object

When you do not need to initiate a call you need to destroy the previously created Session Object to prevent memory leaks.

void wx_voip_session_destroy(wx_voip_session_t session)

parameter

# 7. call

Header file:wmpf/voip.h。

Successfully created. wx_voip_session_type_t Object, you can initiate further calls. call wx_voip_session_call() Initiate a call, call wx_voip_session_hangup() Actively end the call. The call may also be hung up by the other party, or be hung up abnormally, and related events occur by creating session To listen to the callback that you set.

# 7.1 Initiate a call

Call an asynchronous function wx_voip_session_call() Initiate a call:

wx_operation_t
wx_voip_session_call(wx_voip_session_t session,
                     const wx_voip_member_t* callee)

Notes:

1. Before calling this function to initiate an audio / video call, you need to wx_device_is_registered Function to determine whether the device has been registered (a one-time judgment, not multiple judgments). Only on the premise that the device has been successfully registered, the device can call this function to make audio and video calls of the Mini Program.
2. To call the WeChat user's OpenID From the Mini Program that must be used when registering the current device AppID Consistent.
3. Since the WeChat user who is dialed by the mini program audio and video call needs to authorize the device in advance in the mini program, the device returns the WXERROR_PERMISSION_DENIED The user should be prompted to dial does not have permission to make Mini Program audio and video calls.
4. The device side can be accessed via the wx_voip_session_can_call Function to determine whether audio and video calls can be made to the specified user (containing only SDK Interface, there is no background interface).
5. The Mini Program side can be accessed through wmpfVoip.getIotContactList Function to determine audio and video call authorization.
6. We recommend the manufacturer to maintain the backend OpenID With Equipment ID The authorization correspondence.
7. When debugging, manufacturers should use the trial version or development version of the Mini Program, because these versions contain the audio and video call experience duration by default. And the official version of the Mini Program needs to be sent after the equipment can be approved online audio and video calling capabilities.

parameter

Return value

wx_operation_t wx_voip_session_call()Is an asynchronous function that initiates a network request. This asynchronous function returns success, indicating that the call was dialed correctly, not that the call was connected. Note the status code after the asynchronous operation completes.

callee Dxplaination of parameters

The Mini Program side (call receiver) wx_voip_member_t The settings are as follows:

# 7.2 Actively end call

wx_operation_t
wx_voip_session_hangup(wx_voip_session_t session,
                       wx_voip_hangup_reason_t reason)

parameter

Return value

wx_operation_t wx_voip_session_hangup()Is an asynchronous function that initiates a network request. This asynchronous function returns success, indicating that the call was successfully terminated. Note the status code after the asynchronous operation completes.

# 7.3 Mobile phone WeChat call device - Receiving calls

wx_operation_t wx_voip_listener_Join(wx_voip_session_t session,
                                           const char *roomid)

parameter

Return value

wx_operation_t wx_voip_listener_Join()Is an asynchronous function that initiates a network request. This asynchronous function returns success, indicating that the call joined normally. Note the status code after the asynchronous operation completes.

# 7.4 Mobile phone WeChat call device - Hang up early

This interface is only used when creating session Before use, generally used to achieve busy line and reject. create session After, should be used wx_voip_session_hangup Hang up.

WX_API wx_operation_t wx_voip_listener_hangup(const char *roomid,
                                              wx_voip_hangup_reason_t reason)

parameter

Introductions This interface does not require Session, in the wx_init It can then be called directly. If it has been created. Session, you need to use the wx_voip_session_hangup To hang up.

Return value

wx_operation_t wx_voip_listener_hangup()Is an asynchronous function that initiates a network request. This asynchronous function returns success, indicating that the call was successfully terminated. Note the status code after the asynchronous operation completes.

# 7.5 Get Call Room Number roomId

Developers can use roomId Connect the receiving and dialing parties, generally used to identify a call and report the use of information.

const char* wx_voip_session_get_roomid(wx_voip_session_t session)

parameter

Return value

Returns the current voip The identification of the room in which the call was made, i.e. roomid。Or return an empty character string.

# 8. Stop SDK

Header file:wmpf/wmpf.h。

When you no longer use SDK The functionality provided, and you want to free up resources, you can call wx_stop。

Before calling this function, be sure that all previous wx_operation、wx_voip_session Such objects have been released, otherwise unknown consequences may occur.

After successfully calling this function, how do you want to use it again SDK, then you need to go through all the processes from the beginning.

wx_error_t wx_stop()

# Return value

wx_error_t Error code

# 9. Common problem

# 9.1 SDK Link failure

If your device chip is RV1109 And other series chips, this chip provides the tool chain of the libstdc++.so Missing version information, resulting in links to the information we provide SDK When, appear unable to find C++ Problems with standard library related symbols. WeChat side do not understand the tool chain to libstdc++.so If there are any changes, you should seek technical support from the chip manufacturer. For example, replace the tool chain Libstdc + +.so (please remember to replace the system's libstdc++.so）。

# 9.2 device/key/get fail: 10008

That means you're calling wx_device_register Function passed in when the SNTicket Incorrect or expired.

# 9.3 device/call fail: 9

if SDK The error is reported in the log, which means SDK It has been detected that the WeChat user you are trying to dial does not have audio and video calling permission for this device. You need to check the following:

1. Mini Program through wx.requestDeviceVoIP When authorizing device audio and video permissions for this WeChat user:

• Provided by the Mini Program OpenID Whether and the device side when dialed into the wx_voip_session_call of id Agreement
• Mini Program generation SNTicket When used. SN Whether to initialize with the device side SDK The incoming device when ID Consistent. In order to check whether the Mini Program has correctly obtained the audio and video call authorization of WeChat users, the Mini Program developer can use the wx.getDeviceVoIPList Function to get a list of authorized devices for the current WeChat user (devices ID/SN), in order to check whether the WeChat user is on the specified device (via the device ID/SN Audio and video permissions are granted.

2. Device call wx_voip_session_new() When, whether in ID Equipment was introduced. ID
3. WeChat user cancels audio and video rights authorization for the device

• WeChat users remove Mini Programs from WeChat's "Recent" page
• WeChat user unsubscribes in Mini Program settings page Both of these situations will cause WeChat users to cancel the audio and video authorization for the device. SDK Need to encounter the error (by judging the wx_voip_session_call Whether the return value is WXERROR_PERMISSION_DENIED), the situation should be handled correctly and the device user should be alerted to the situation. For example, remind the device user that "Because the WeChat user has cancelled the authorization of the device, you will not be able to make a call to the WeChat user."

# 9.4 device/call fail: 12

if SDK The error is reported in the log in two cases:

# Mini Program does not open audio and video permissions

If your Mini Program does not open audio and video permissions in the Mini Program background, you need to open it first.

Due to the opening of the audio and video permissions of the official version of the Mini Program, it is necessary to send equipment for review, and for device developers, it is possible to init_Config- >wxa_flavor = WX_WXA_FLAVOR_DEMO The experience or WX_WXA_FLAVOR_DEBUG That is, Mini Program development version. As to which version to use, please consult your Mini Program developers.

If you need to use the Mini Program development version, then the device developer needs to obtain the developer rights of the Mini Program, and scan the real machine of the Mini Program developer's WeChat developer tool to debug the two-dimensional code to download the development version of the Mini Program.

If you need to use the Mini Program experience version, then the device side developer needs to get the experience permission of the Mini Program, please consult your Mini Program side developer.

# SDK Check device security failure

First, you need to check SDK Does the log contain the following:

CallerTicket gained, you now can create voip session for IoT scene

If the above log is not included, the explaination SDK Failed to verify device security.

At this point you should pay attention to check the information you pass to SDK of Data Does the directory exist cert File, rpmb_key Documents and rpmb_Data Documents. You need to make sure Data The directory has enough disk space to hold these files, and these files need to be persistent, not overwritten or deleted during firmware upgrades, and not tampered with. If these files have been tampered with, you may be in danger of experiencing OperateWxData fail: 12 There were other mistakes before.

# 9.5 device/call fail: 13

if SDK This error in the log indicates that you have already initialized the device once. SDK After that, the equipment was replaced. ID。

It is important to note that the equipment ID Must have a one-to-one mapping to the device, and the device on a device ID Should not be changed. If you encounter this error, you should use the device that initialized the device originally. ID to initialize SDK, rather than replacing the device ID。

# 9.6 device/call fail: 20

Incoming OpenID Incorrect, not legal. OpenID。Please ensure that incoming OpenID By Mini Program wx.login Obtained.

# 9.7 device/register: HTTP 0 XX

This means SDK Encountered when registering the device HTTPS Error, here. XX It should be a number, like device/register: HTTP 0 77 is an error code. 77, please. CURL Check the error code corresponding to the error is what, if the error you can solve, such as network connection failure, please solve your own.

# 9.8 device/register: errcode: 9800011 errmsg: sn_Tickle illegal

This indicates that the incoming SNTicket Incorrect.

SNTicket Format is a combination of numbers and letters, the length 32 It contains no other symbols. Be careful not to misrepresent other parameters, such as misinformation access_token。At the same time, in order to avoid devices or Mini programs to obtain AccessToken Cause a security risk, you should provide the server side to generate snticket Interface to Mini Programs and devices.

# 9.9 device/register: errcode: 9 errmsg: device XXX not registered

This means that the given device ID It's already tied to a device. It is important to note that the equipment ID Must have a one-to-one mapping with the device, the device ID Cannot be shared by multiple devices or migrated from one device to another. Nor does it allow one device to replace the device ID。If you do the above, such as continuing to use an already registered device on another device ID, you should switch to another device on this "other device" ID。

# 9.10 [CheckMemoryLeak]: Detected memory leak: wx_operation, leak objects: 1

This error log indicates that you are calling wx_voip_XXX Series of functions, or others that return wx_operation Object, its return value is not processed. In a call that returns the wx_operation Object, you must call its return value wx_operation_await, wx_operation_wait or wx_operation_destroy To wait asynchronously for the operation to complete, synchronously for the operation to complete, or not wait for the operation to finish, to release the wx_operation Object.

# 9.11 Found local timestamp incorrect, expected XXX, actual XXX

This error indicates that your system timestamp is incorrect. Make sure that the time stamp of your device system does not differ from the current time by more than 5 Seconds.

SDK adopt Time(NULL) Function to get the system timestamp, Since the time stamp does not have a time zone, Please make sure that you get the timestamp in this way and UTC Time match.

# 9.12 The camera image collected by the display device in the Mini Program is rotated 90 degree/Screen stretching deformation

Due to the Mini Program VoIP The historical legacy, in WeChat 8.0.40 And the following versions, the Mini Program shows VoIP Image will rotate the received video content when the 90 Degree. Mini Program VoIP The plugin provides setUIConfig Interface allows the manufacturer to solve the rotation 90 Camera Rotation and aspectRatio.

if (/* WeChat version <= 8.0.40 */) {
  wmpfVoip.setUIConfig ({
    // When setting up a video call caller Frame
    callerUI: {
      cameraRotation: 0,  // The angle of swing of the 0, 90, 180, 270。default 0
      aspectRatio: 4/3, // Aspect ratio, the caller's video screen will be adapted to the proportion, the effectiveness Numbers. default 4/3
      horMirror: false, // Horizontal mirror, boolean Value, the default false
      vertMirror: false, // Vertical Mirror, Ibid.
      enableToggleCamera: true, // Does it support toggling the Portrait, false Does not showCamera switchButton. Default false [This configuration item is invalid in wmpf, wmpf is turned on by default, and does not display the on / off button]
    },
  })
}

For WeChat 8.0.41 And above, Linux SDK Support for the adoption of wx_Camera_stream_listener::data Function parameter rotation Set to WX_VIDEO_ROTATION_0 Or other to specify the angle of swing.

# 9.13 [wx_operation_validate]: invalid address: wx_operation 0xXXXXXXXXX

Means you're calling wx_operation_wait、wx_operation_await、wx_operation_destroy In the same function. wx_operation The object address is incorrect or the object has been released. Please check the code for correctness.

# 9.14 Call wx_Camera_stream_listener::data with stream 0xXXXXXXXX, user_Data 0xXXXXXXXX. But it's released.

Means you're calling at the wrong time. wx_Camera_stream_listener::data Function. Needs attention wx_Camera_stream open Before you can start calling listener::data functionin wx_Camera_stream close Then you can't call listener::data Function. You also need to pay attention to thread safety issues. If you maintain through a list wx_Camera_Stream, then when traversing the list, you need to interact with the open、close Lock together to avoid data competition.