# Weixin Mini Program Audio Video Calling + Camera (for hardware)

Weixin Mini Program Audio and Video Calling (for hardware) Manual

# 1. Product Description

# 1.1 Weixin Mini Program Voip

With the WeChat Weixin Mini Program audio and video call plug-ins hardware developers can use the Mini Program hardware frameworks (WMPF) 、Linux SDK Access and cloud access are three ways to achieve one-to-one audio and video calls on smart devices and mobile phone WeChat, meeting real-time touch scenarios and improving the call experience.

The following diagram is a demonstration of the mobile phone. Authorized pop-ups, call reminders, and call interfaces are the unified interface provided by WeChat. The hardware Weixin Mini Program can implement the above functions when it plugs into WeChat VOIP call plug-in.

It is suitable for a variety of devices and scenarios such as campus voice phones, access control machines, smart door locks, smart control screens, smart TVs, smart cameras, smart speakers, smart retirement and so on, supporting two-way calls on hardware devices and mobile phones to achieve strong call reminders.

# 1.2 Weixin Mini Program Portrait

With the [capability in the Weixin Mini Program audio and video call plug-in, developers can quickly connect the camera to the camera class.WeChat Weixin Mini Program ecosystem ( Cloud-to-Cloud approach]] ), user viewing camera streaming media is also more convenient, cultivating the habit of user Mini programs watching cameras.

For the functions involved in the camera, it is divided into components optional, components optional and custom.

type	Modular	Functional points	describe
Component optional	Player	play	Play, pause
Component optional	Player	Image Zoom	Zoom in and out
Component optional	Player	volume	Volume, muteness
Component optional	Player	screen	Small screen, full screen
Component optional	Speaking to you	The device calls the cell phone	VoIP features
Component optional	Player	definition	Developers come in
Component optional	Speaking to you	Mobile phone calling devices	Call, hang up.
Customization (for example)	roulette	Directional Control	Generally used to control the turn of the camera
Customization (for example)	Value-added services	Cloud Storage
Customization (for example)	Value-added services	AI services

Weixin Mini Program Interface :

Full screen interface :

# 2. Equipment requirements

Currently supports Android, Linux, RTOS devices. Each device can only bind one Weixin Mini Program and can only talk to one Mini Program.

At present, we support two access modes: "device direct connection" and "cloud to cloud":

Direct device connection: Supports Android and Linux devices .The device sends and receives audio and video streams and signaling directly to the WeChat background, and calls with Weixin Mini Program in the user's mobile phone WeChat.
Cloud-to-cloud: Support for low-power Linux, RTOS and other system devices that cannot meet the direct hardware needs of the device.The device side only performs device verification, and the developer background (the service side) forwards audio video streaming and messages to the WeChat background, and makes calls with Weixin Mini Program in the user's mobile WeChat.

# 2.1 Device Connected (Android)

Android device side run Weixin Mini Program hardware framework (WMPF), WMPF run developer Mini Program, directly with the user mobile phone WeChat Mini Program to talk;

The equipment shall meet the following basic requirements:

Has audio and video capabilities (microphones, cameras, etc.);
Meet the equipment requirements for Weixin Mini Program equipment certification ;
Meet Android Weixin Mini Program hardware framework running requirements
- System version: Android 7.1 and above is recommended
- CPU: at least quad-core 2GHz, video calls have higher requirements on the CPU
- Memory: At least 2GB RAM + 8GB ROM
Weixin Mini Program hardware framework v2.1.0 and above is recommended.

# 2.2 Direct Device Connection (Linux)

Linux system device-side call Weixin Mini Program Audio Video Calling SDK (Linux) , Directly to the user's mobile phone WeChat in the Mini Program calls.

The equipment shall meet the following basic requirements:

Has audio and video capabilities (microphones, cameras, etc.);
Meet the equipment requirements for Weixin Mini Program equipment certification ;
Meet the device requirements for running the Weixin Mini Program Audio and Video Calling SDK (Linux) .

# 2.3 Cloud-to-Cloud

Currently supports some low-power Linux, RTOS and other systems of equipment.The device side of the device verification, through the developer background transfer, and the user's mobile phone WeChat Weixin Mini Program to talk.

The equipment shall meet the following basic requirements:

HTTPS communication capability
Storage capacity
Audio (visual) capabilities

# 3. Pre-development preparation

The following steps can be carried out simultaneously and involve multiple platform-side audit processes, so please allow time in advance.

# 3.1 [Android devices only] Access WeChat hardware platform

Registration WeChat terminal cooperation platform and WeChat open platform account, registration of equipment information and other links involve platform audit, please prepare in advance.

Refer to the documentation and guidance for completing the process of registering the WeChat endpoint cooperative platform (wecooper), binding the Mobile App and registering the hardware.

This step is to connect the device to the device system of WeChat, complete the association between the device and the APP, and the APP and Weixin Mini Program.

To avoid confusion, the Mobile App appId is generally referred to as hostAppId;
Each hardware device needs to register with the hostAppId identity calling the addDevice interface to use Android WMPF.

# 3.2 Device access and ability to apply for devices

This process involves platform auditing, so please prepare early.

Weixin Mini Program To use the voice and video calling capabilities, you need to apply to open the "voice and video capabilities" device capabilities in the Mini Program management platform **** .See the Device Access document and Weixin Mini Program Audio-Video Calls (for hardware) Use the process guidance in the manual and pay attention to [Hardware VoIP Audit Verification Requirements]]](https://developers.weixin.qq.com/community/minihome/doc/00002e131e8cc8ae8a7f5473f56c01) .

Once the access is complete, the developer gets a model_id assigned by the platform.Model_id corresponds to a device type and is an important credential for invoking the Weixin Mini Program device capability related interface.

After obtaining model_id, developers can call the interface to obtain the device ticket to obtain snTicket for the subsequent device validation process.

# 3.3 Access VOIP plug-in

The main functions of Weixin Mini Program audio-visual calls are provided through the VOIP Calls Mini Program plug-in [(appId: wxf830863afde621eb).

In Weixin Mini Program management background to complete the "Mini Program audio and video capabilities" application and through, the Mini Program can directly use the VOIP call plug-in, without additional application.

If a developer wants to debug ahead of time, they can apply manually: Log in to the Weixin Mini Program administration background - Settings - Third party settings - Plugin management, click Add Plugin, search for and add the VOIP call plugin.

# 4. Device-side development

The Android device side needs to run an Android app provided by the developer to register the device and run Weixin Mini Program for VOIP calls.Please refer to < Developing device-side applications (Android) >
Linux device side needs integration Weixin Mini Program Audio and Video Calling SDK (Linux)
The RTOS device side needs to integrate the SDK for making calls.

# 5. Weixin Mini Program Development

Developers need to develop (or use an existing) Weixin Mini Program plug-in for "VOIP Calls" to enable the ability to make and receive audio and video calls.

When making calls with Android devices, developers need to use the same Weixin Mini Program, running both on the Android device (the device initiates or receives the call) and on the mobile WeChat client (the mobile user initiates or answers the call).
When calling with a Linux device, the device side runs the Weixin Mini Program Audio-Video Calling SDK (Linux) and the mobile side runs a Mini Program inside the WeChat client (the mobile user initiates or receives the call).
When calling an RTOS device, the device-side SDK is used to initiate the call, the service-side sdk is used to stream video through the process, and the mobile-side WeChat client runs Weixin Mini Program (the mobile user initiates or receives the call).

# 5.1 Core processes

Use at least WeChat Client 8.0.30 and above. It is recommended to use the latest version.

Weixin Mini Program The development of the following links, please refer to the documentation of each link:

Access the "VOIP call" plug-in: Refer to plug-in documentation to introduce the plug-in in Weixin Mini Program;
Device Calling Cell Phone WeChat:

The user needs to authorize the device at the end of the WeChat mobile phone first, please refer to [User Authorization]]](./voip/auth.md) ;
It is suitable for devices that are registered and authorized by the user to initiate a call. The user receives the call within the cell phone WeChat, please refer to "Device Call Mobile WeChat" ;

Mobile phone WeChat calling device: It is suitable for the user to initiate a call in the mobile phone WeChat, and the device that has been registered and authorized by the user to answer,Please refer to [for Mobile WeChat Calling Devices (Android]]](./voip/call-wmpf.md) and [for mobile WeChat Call Devices (Linux]]](./voip/call-device.md) ;
Performance and Experience Optimization: Refer to Performance and Experience Optimization .

**Call-related anomalies, please refer to "Call Anomaly Troubleshooting Guide" **

# 5.2 Debugging instructions

Debugging in the WeChat developer tools is not supported for VoIP call flow. Please use a real machine.

# 5.2.1 Device side using "development version / experience version Weixin Mini Program"

Android device side can specify the use of "development version / experience version Weixin Mini Program" for debugging. Please refer to "Developing device-side applications (Android)" "3.2 Run development / experience version of Mini Program."

# 5.2.2 Use the development / experience version Weixin Mini Program to answer calls

When the receiver receives the message push and clicks on the answer, the default is the official version of Weixin Mini Program.

During the development phase, it is recommended to call wmpfVoip.initByCaller An additionalminiprogramStateparameter is passed to specify the open development / demo version Weixin Mini Program.

const result = await wmpfVoip.initByCaller({
  // 其他参数省略
  miniprogramState: 'developer', // formal/正式版（默认）；trial/体验版；developer/开发版
})

When using the Weixin Mini Program Audio and Video Calling SDK (Linux), you can set wxa_flavor at initial build to specify that calls are answered using the development / demo version.

NOTE:

When the receiver is a mobile phone WeChat, you should scan the development version of the two-dimensional code generated by the WeChat developer tool in advance to download the development version.
When the receiver is a device running WMPF, you should download the development version in advance by following the steps in 5.2.1.
You can use wx.getAccountInfoSync Interface determines the current Weixin Mini Program version.

# 6. Service side development

When RTOS devices use a cloud-to-cloud solution, they need service development .Direct device connection (Linux, Android) does not need to be developed.

# 7. Common problem

Please refer to Frequently Asked Questions (FAQ)