In today's application market, it is difficult for traditional application products to fully meet the diverse needs of consumers. In order to stand out in the competitive market, enterprises need to gain deep insights into user needs and provide personalized service experiences and differentiated product innovations to attract and retain consumers.
For example, shopping apps can realize virtual dressing by combining real and virtual; interior design apps can place virtual furniture models in the real environment to give users a demonstration; gaming apps can place virtual game screens in the real environment to provide users with a more immersive gaming experience.
The above innovative application scenarios all rely on a core technical capability, namely augmented reality (AR) technology, the HarmonyOS SDKAR Engine Services(AR Engine) has just that capability.
AR Engine is an engine for building augmented reality applications on HarmonyOS. Based on the combination of real-time computer computation and multi-sensors, it can realistically superimpose virtual objects onto real scenes through the device's camera, allowing the application to realize the fusion of the virtual world and the real world, and to provide users with a brand-new visual experience and interaction method.
AR Engine provides spatial computing capabilities such as motion tracking, environment tracking, and hit detection, with which developers can realize applications such as AR navigation, AR measurement, and AR placement, providing users with a new interactive experience that blends reality with reality.
Motion tracking capability
Motion tracking capability unifies the virtual coordinate system and the real coordinate system, and is the basic capability to realize the realistic integration of virtual objects with the real environment.
AR Engine uses motion tracking capability to identify feature points through the end device camera and track their movement changes, and combines the movement changes of these points with the inertial sensors of the end device to continuously track the position and attitude of the end device. By continuously tracking the device's position and attitude change trajectory relative to the surrounding environment, a unified geometric spatial coordinate system of the virtual digital world and the real physical world is established, thus realizing that the virtual object maintains the consistency of the geometry with the real environment when the device is in a different viewpoint and distance, and the developer can superimpose the rendered virtual object to the camera image from the observer's viewpoint to achieve the integration of the virtual and the real.
For example, when AR Engine is applied to an AR car show, users can rotate the device's camera to view the virtual car model from different angles, realizing an immersive AR view of the show, where the ability of motion tracking is used.
Environmental tracking
AR Engine environment tracking capability can quickly identify the plane information in the real scene and recognize the corresponding plane type and shape and other information, thus assisting the application to realize the digital virtual objects to be integrated into the real physical world in a scene-oriented way. Environment tracking mainly includes three capabilities: plane detection, plane semantics, and target semantics.
Plane Inspection
The plane detection capability can obtain information about planes in the real world, such as floors or walls, by recognizing clusters of feature points in the horizontal and vertical planes, as well as the boundaries of the planes. Once these planes are detected, applications can place virtual objects based on the planes or develop application scenarios such as navigation.
planar semantics
Plane semantics refers to detecting the types of real-world planes, and there are 8 types of planes currently supported, namely: wall, floor, seat, desktop, ceiling, door, window, and bed.
target semantics
Target semantics refers to detecting the shape of a target object when it is located on a flat surface, which currently includes rectangles and circles.
hit detection
The hit detection technology can help AR Engine realize the connection between two points in the virtual and real worlds. AR Engine obtains the mapping relationship between points of interest to the points of interest in the AR world through the hit detection technology, and developers can help users realize the interaction with virtual objects in the AR world based on this ability.
Schematic of Hit Detection Capability
In the actual application scenario, the user can click on the terminal equipment screen, select the point of interest in the real environment, the AR application calls the collision detection interface to obtain the point of interest of the AR world and return the relevant information, the user can complete the model placement and other interactive actions on the point of interest of the AR world displayed on the screen of the terminal equipment, for example, the user can be placed in the real indoor environment through the camera to place the virtual furniture and observe and judge whether the furniture is in line with the overall design from all angles, thus reducing the error of purchasing large items. For example, users can place virtual furniture in the real indoor environment through the camera, and observe and judge whether the furniture conforms to the overall design from various angles, thus reducing the error rate of purchasing large items.
Motion tracking capability unifies the coordinate system of the real world and the virtual world, environment tracking capability identifies the plane information of the real world, and hit detection capability establishes the interaction anchor point between the two worlds. AR Engine breaks down the barriers between the physical world and the digital world in multiple dimensions from the point, line, and surface through these three capabilities, realizing the overlay and unification of the two, which not only provides an unprecedented experience for users, but also opens up a wide innovation space for developers, bringing more possibilities for various industries. This unification not only provides users with unprecedented experience, but also opens up a broad innovation space for developers and brings more possibilities for various industries. With the continuous evolution of the technology and the expansion of the market, we can expect AR technology to have a wider range of applications and a more far-reaching impact in the future.
Learn more >>
interviewsAR Engine Alliance Official Website
gainAR Engine Development Guidelines