Research Article - (2022) Volume 11, Issue 1
Marker-less motion capture is a rapidly advancing field that can take simple RGB image sequences, or more advanced Red Green Blue Depth (RGB-D) image sequences obtained using depth sensors, and outputs an estimated human pose. This method of human pose estimation allows for the extraction of biomechanical features which can then be analysed by clinicians to give more insights into a patient’s movement capabilities. When compared to other, more clinically proven technologies such as the Knee Kinesiography (KneeKG), biomechanics presented have the advantage of being more representative of natural movement without the obstructive markers placed on the body. This Significant difference of up to 10 degrees in a range of motion for the knee could be the key to better identifying a person’s gait or tracking their natural walking pattern over time, while also being more robust and better suited to a smaller clinical environment.
Human motion analysis • artificial intelligence • Computer-Aided Diagnosis (CAD) • Automated Rehabilitation (AR) • Knee Osteoarthritis • deep learning
In the UK, both in the NHS and in private healthcare, an estimated 15% of all patients who consult their doctors do so for movement-related issues; resulting in a large cost to the industry in terms of time, money, and manpower [1]. Through a series of studies, the prevalence of knee osteoarthritis is estimated to be 19.2% in adults ≥ 45 and 37% in adults >60; this shows the reason behind working towards cheaper and faster diagnosis [2]. Due to the individuality of movement debilitating issues and the nature of pain tolerance, for example, anterior knee pain may not present itself with any measurable abnormalities with movement and people may suffer from movement issues without any noticeable pain [3,4].
Another large impact on the healthcare industry is the cost of treatment, especially in the case of surgical treatment. For example, one commonly reported movement issue which frequently arises is a meniscal tear, these can also occur in cases without knee pain, it has been reported that the global annual cost for an arthroscopic partial meniscectomy is $4 billion to treat these meniscal tears [5]. This large cost has increased the number of studies focusing on the effects of non-surgical treatment such as physiotherapy. One such study gathered participants with osteoarthritis of three different pain levels, as measured by the WOMAC score, which after a four-week rehabilitation process all resulted in a WOMAC score reduction of between 5.9 and 7.7 [6].
Computer-Aided Diagnosis (CAD) and Automated Rehabilitation (AR) are important for the future of healthcare to allow for a more objective and faster diagnosis, while also providing a more robust and personalised diagnosis using these objective quantifications rather than the standard subjective markers [7]. These ideas of CAD have often been considered the future of medical diagnosis, however, in the past, the techniques and hardware were not capable of the predictive accuracy required [8]. In addition, AR systems provide the capabilities to better understand a problem and create a personalised treatment plan specific to a patient’s underlying issues and the presented movement issues [9].
The aim is therefore to investigate the uses of state-of-the-art computational approaches to marker-less human pose estimation for human motion analysis, that provide a more in-depth look at how the human body functions. This method will allow for more objective measurements performed by artificial intelligence that will help to both diagnose movement issues, such as Osteoarthritis, and provide a personalised rehabilitation strategy that tackles the individuality of the patient’s problems.
The traditional methods for human motion analysis using markerbased fixed-position multi-camera systems, such as the Vicon Motion Systems, though this is a more expensive option the results are considered to be the real ground truth due to their accuracy [10]. However, the development of low-cost alternatives to these marker-based methods has been at the forefront of research to provide wider access to these powerful techniques. One technique uses augmented reality markers placed on various body parts which are tracked using a standard Red Green Blue (RGB) camera, one issue with this technique is that the tracking accuracy depends on both the placement of the markers and cameras [11]. Another more specific technology is the Knee Kinesiography (KneeKG) which uses an IR sensor to track markers placed on and around the knee, however, this technique is limited to just one leg at a time and can only measure gait while using a treadmill [12].
For a smaller, more portable technology for human pose estimation (HPE) and human motion analysis recent research has been moving towards Red Green Blue Depth (RGB-D) cameras which are a dual-camera system with an RGB camera and a near-IR depth sensor that can be used to directly predict the joint positions and their 3D position with just one small and relatively cheap device [10,13]. One other use for portable devices such as the Azure Kinect is their use in distance rehabilitation, providing more non-surgical treatment options for physicians to offer with the added benefit of ensuring the patient sticks to the regime by monitoring their exercises remotely [14]. Given that the Kinect devices began as a new option for gaming-related controls; another added benefit would be that this technology has been in and around the field of human-computer interactions, leading to the gamification of homebased rehabilitation options which has found a greater success rate than standard rehabilitation regimes [15].
Hardware
Azure kinect (Microsoft, Washington); -This camera is a dual function camera with both RGB capabilities and a near-IR time of flight depth sensor to form an RGB-D image, with an open-source SDK for both the sensors and native marker-less body tracking support [13]. In addition, the cameras support the ability to run multiple devices, in varying configurations, to capture the same object which reduces the occlusion at the cost of higher computing power for the body tracking. To aid the data collection the native body tracking produces the global cartesian coordinates of joints and the confidence level of each joint [10].
Each coordinate represents the predicted 3D location of a joint as seen by two Kinect devices, with each map being used for point registration. This graph, as seen in (Figure 1) also has a silhouette of a person overlaid on top of the graph to show which joints are represented. These joints are produced with a network of convolutional neural networks (CNNs) to process a combination of RGB images and depth maps, trained using a combination of synthetic and real images. The resulting skeleton maps are fit onto a depth map to estimate the real 3D coordinates and the orientation of each predicted joint.
Figure 1: Example of the data collected by the Azure Kinect system, with a silhouette projected on top of the graph to help show what each point represents on the body. With each point representing a joint on the body, and each shape represents a different Kinect’s joint prediction where both are used to determine the real 3D positions of each joint.
Knee kinesiography: This device, on the other hand, uses an infrared (IR) motion capture system to track markers placed around the knee and on the leg, classified as a non-invasive, easy to use a system that does not suffer from skin artefacts due to the markers not being placed on the body and instead use relative locations on the apparatus on the legs [12]. The KneeKG (Emovi, Montreal) system also offers a commercialised software allowing for visualisation of the leg showing the flexion, abduction, and rotation in real-time, along with a generated report showing outlying the gait results intending to show the areas to target for physiotherapy or surgery [16].
Device set-up: The experimental set-up used a treadmill, to collect the walking data, along with the KneeKG and two Azure Kinect devices to record the movement. The computer used to record the data collection was a portable high-end laptop with a GeForce RTX 2070 (Nvidia, California) graphics card, this recorded the skeleton data from two Azure Kinect devices at an average of 12fps with the colour exposure time set to 10000.
Data collection
First, the subject was asked to walk on the treadmill for five minutes to be comfortable walking on the treadmill and to discover their comfortable walking and fast walking speeds. Once comfort was achieved the two Azure Kinect devices with the real-time joint prediction recorded the subject walking for three repeats each lasting one minute, for both the comfortable (3 km/h) and fast (3.5 km/h) walking speeds. After these were recorded the markers for the KneeKG device were placed on the left leg and the trial was repeated, first walking for five minutes to regain the comfortable level then the Azure Kinect recorded the movement with the KneeKG device on the leg.
Biomechanics of movement were then calculated, focusing on the knee flexion and the hip abduction for each collection time point. This spatiotemporal analysis allows comparisons to be made between the two techniques, with the initial experiment outlining the change in range of motion in each plane of movement. These biomechanics were calculated using a purpose-built data analysis pipeline, which can use the raw movement in a cartesian coordinate system to calculate velocity-based and angular-based biomechanics. Velocity-based calculations are simpler, with very established formulas and simple functions using information regarding a single joint. Angular equations, on the other hand, have adapted the cosine rule, given any three joint locations and the distances between the joints the angle can be calculated using eqn.1 where a, b, and c represent the distances between the given three joints and γ represents the angle being calculated.
These results of the biomechanics calculations can be seen in (Table 1), with the primary focus being on knee flexion and hip abduction. When the subject was wearing the KneeKG device the range of motion for knee flexion was reduced by up to 10 depending on the walking speed, while the hip abduction had no significant changes. This table can also be represented as a graph, as seen in (Figures 2 - 5) showing the flexion of the left knee during the entire recorded walk, this is useful when identifying anomalies in how a person walks or identifying the walking pattern. Only the left knee is displayed due to the KneeKG only allowing one leg to be analysed at a time, the right knee would therefore not be affected.
| Flexion Range (L) | Flexion Range (R) | Abduction Range (L) | Abduction Range (R) | |
|---|---|---|---|---|
| Marker-less (3km/h) | 71.25° (± 6.83) | 77.04° (± 2.93) | 16.23° (± 1.20) | 17.63° (± 0.68) |
| Marker-less (3.5km/h) | 68.76° (± 2.09) | 70.02° (± 4.81) | 16.54° (± 1.00) | 18.90° (± 0.97) |
| KneeKG (3km/h) | 56.97° (± 2.61) | 72.79° (± 1.56) | 16.83° (± 0.36) | 17.41° (± 0.37) |
| KneeKG (3.5km/g) | 59.94° (± 1.81) | 67.18° (± 1.98) | 16.23° (± 0.43) | 16.55° (± 0.62) |
Table 1. Range of flexion and abduction for both left and right legs, measured at a normal (3 km/h) and fast speed (3.5 km/h). Also showing both with and without the KneeKG alongside the standard deviation specified in parentheses. Range of Flexion/Abduction angles were calculated using the movement data collected from a single, healthy subject, walking on a treadmill for one minute and each had thr ee repeats.
Figure 2: Graph showing the knee flexion over an entire video sequence of a subject walking on a treadmill (at 3 km/h) with the KneeKG device attached to the leg, the knee flexion was calculated using eqn.1 on the data collected by the Azure Kinect devices. This graph shows an overall reduction in the max flexion and a less stable walking pattern given the differences between the flexion of each step.
Figure 3: Graph showing the knee flexion over an entire video sequence of a subject walking on a treadmill (at 3 km/h) without the KneeKG device attached to the leg, the knee flexion was calculated using eqn.1 on the data collected by the Azure Kinect devices. This graph shows a normal walking pattern with increased maximum knee flexion, the walking pattern itself still has abnormalities however overall is more stable.
Figure 4: Graph showing the knee flexion over an entire video sequence of a subject walking on a treadmill (at 3.5km/h) with the KneeKG device attached to the leg, the knee flexion was calculated using eq.1 on the data collected by the Azure Kinect devices. This graph shows an overall reduction in the max flexion and again, a less stable walking pattern giv en the differences between the flexion of each step.
Figure 5: Graph showing the knee flexion over an entire video sequence of a subject walking on a treadmill (at 3.5 km/h) without the KneeKG device attached to the leg, the knee flexion was calculated using eqn.1 on the data collected by the Azure Kinect devices. This graph shows a normal walking pattern with an increased maximum knee flexion while also having periods of lower minimum knee flexion, the walking pattern is the most stable with the fewest abnormalities.
The knee flexion at a comfortable walking speed of 3 km/h is seen in Figures 2 and 3, both with and without the KneeKG respectively. The key differences between these two are the increased maximum flexion when the subject does not have the markers placed around the knee.
Knee flexion at a faster walking speed of 3.5 km/h shown in given (Figures 4 and 5), both with and without the KneeKG device respectively. This also shows the increased maximum flexion while also showing that without the KneeKG device the subject has a more stable walking pattern.
When comparing the two motion analysis techniques, it is important to understand the statistical significance of the results. A paired t-test was used to compare each time point for both techniques, resulting in a p-value < 0.05 for both knee flexion and hip abduction. This shows that markerless human motion analysis is significantly more representative of natural walking since there are no restrictions to movement.
These results show the potential for a marker-less human motion analysis method in a clinical setting, due to both the efficiency of the collection method and the results being more representative of natural human movement. Which can also provide a simpler and more in-depth method of analysing and discovering new biomechanical biomarkers. In terms of the natural representation of movement, the increase of 10-15° in the range of flexion is a significant difference when it comes to the simple task of walking.
This would be beneficial when using the movement data to make decisions, if the flexion during walking is altered it would be difficult to determine whether a flexion deficiency is due to an underlying issue or the data collection technique. One limitation to these results, however, is that the method was only performed on a single subject due to the limited participant pool. This did not significantly alter any results since the participant was a healthy control with no signs of any movement issues, therefore, any results obtained will still be representative and because of the differences between the two methods however further research to include more participants would be beneficial.
Given the open nature of the Azure Kinect and the amount of data provided, in terms of spatiotemporal parameters, there are a wide variety of features that can be extracted regarding movement. This ranges from simpler calculations such as the force of a movement and, the moment of the leg during the swing to more complicated calculations such as the effects of angular momentum on the knee joint during gait, and the net forces for the entire musculoskeletal model combined with the interactions from each muscle to each connected bone [17-19].
In terms of efficiency, the time per data acquisition for the alternative technique of the KneeKG is 15-20 minutes when performed by a trained technician [12]. However, when performed by a clinician with limited experience the acquisition time averages at 30 minutes. When this is compared to the acquisition time for the Azure Kinect, since the set-up is only performed once throughout the acquisitions, the acquisition time per person is equal to the time on the treadmill. Unlike more commercially available biomechanics tools, the Azure Kinect has no requirements for the size of the room or the need for a treadmill. This same method can be used on a series of clinically significant exercises, rather than using gait analysis, to provide a more in-depth analysis of a patient when performing a physical exam.
Another benefit of using a low-cost alternative is the potential of proving patients with an option for distance rehabilitation, with increased benefits from the advances of gamification which leads to more engagement with the treatment plan [14,20]. Along with distance rehabilitation benefits, access to this hardware allows for more availability for longitudinal data collection. This will therefore allow the clinician to assess the patients’ needs more accurately through the change in the patient’s biomechanics over time.
Citation: Armstrong K, et al. Novel Clinical Applications of Marker-less Motion Capture as a Low-cost Human Motion Analysis Method in the Detection and Treatment of Knee Osteoarthritis. J Arthritis, 2022, 11(1), 001-005.
Received: 14-Dec-2021, Manuscript No. 49786; Editor assigned: 16-Dec-2021, Pre QC No. 49786; Reviewed: 15-Jan-2022, QC No. 49786; Revised: 23-Jan-2022, Manuscript No. 49786; Published: 25-Jan-2022, DOI: 10.4172/2167-7921.2022.11.053
Copyright: 20 22 Kai Armstrong, et al. This is an open - access article under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
