3D Modalities Research Articles

Accurate prediction of disease-risk factors from volumetric medical scans by a deep vision model pre-trained with 2D scans.

The application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities. The model, which we named SLIViT (for 'slice integration by vision transformer'), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound). SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications.

Tracking 3D motion of instruments in microsurgery: A comparative study of stereoscopic marker-based vs. deep learning method for objective analysis of surgical skills

Background:In microsurgery, integrating motor and visual information is crucial for achieving precise bimanual maneuvers. Computer vision serves as a useful tool for analyzing surgical maneuvers and enhancing situational awareness in medical assistive systems. Furthermore, the combination of deep learning with computer vision shows promise for objectively assessing surgical skills. Methods:This study compared two methods for tracking 3D movement during microsurgery: Mitracks3D, a stereoscopic tracker using color-marked instruments, and a deep learning method for instruments identification based on YOLOv8. Additionally, drawing an analogy between single-camera and new stereoscopic recording systems, 2D and 3D motion signals from microsurgery training videos on the peg transfer task were obtained to analyze surgeons’ psychomotor skills using six motion analysis parameters (MAPs). Results:When comparing the two 3D tracking methods, the motion signals revealed a relative error (ϵ) of less than 10%, with a correlation coefficient (ρ) exceeding 0.9892 ±0.0020, tracking loss below 2.0398%, and instruments detection time under 1 ms for Mitracks3D and 7.09 ±0.62 ms for YOLOv8. When comparing the performance between participants using 2D and 3D signals, statistically significant differences were found in 4 of the 6 explored MAPs. Additionally, a high cosine similarity (cs>0.99) was found between the performance scores of the 6 MAPs, indicating compatibility between the 2D and 3D recording modalities. Conclusions:The deep learning-based approach is as effective as stereo-tracking with markers; both methods produced remarkably similar 3D motion signals. The compatibility in evaluation between 2D and 3D was confirmed. In essence, adopting computer vision methods such as Mitracks3D and YOLOv8 provides a compelling alternative for constructing objective assessment tools for surgical skills.

Measurement precision bounds on aberrated single-molecule emission patterns

Single-molecule localization microscopy (SMLM) has revolutionized the study of biological phenomena by providing exquisite nanoscale spatial resolution. However, optical aberrations induced by sample and system imperfections distort the single-molecule emission patterns (i.e. PSFs), leading to reduced precision and resolution of SMLM, particularly in three-dimensional (3D) applications. While various methods, both analytical and instrumental, have been employed to mitigate these aberrations, a comprehensive analysis of how different types of commonly encountered aberrations affect single-molecule experiments and their image formation remains missing. In this study, we addressed this gap by conducting a quantitative study of the theoretical precision limit for position and wavefront distortion measurements in the presence of aberrations. Leveraging Fisher information and Cramér-Rao lower bound (CRLB), we quantitively analyzed and compared the effects of different aberration types, including index mismatch aberrations, on localization precision in both biplane and astigmatism 3D modalities as well as 2D SMLM imaging. Furthermore, we studied the achievable wavefront estimation precision from aberrated single-molecule emission patterns, a pivot step for successful adaptive optics in SMLM through thick specimens. This analysis lays a quantitative foundation for the development and application of SMLM in whole-cells, tissues and with a large field of view, providing in-depth insights into the behavior of different aberration types in single-molecule imaging and thus generating theoretical guidelines for developing highly efficient aberration correction strategies and enhancing the precision and reliability of 3D SMLM.

A Review of 3D Modalities Used for the Diagnosis of Scoliosis.

Spine radiographs in the standing position are the recommended standard for diagnosing idiopathic scoliosis. Though the deformity exists in 3D, its diagnosis is currently carried out with the help of 2D radiographs due to the unavailability of an efficient, low-cost 3D alternative. Computed tomography (CT) and magnetic resonance imaging (MRI) are not suitable in this case, as they are obtained in the supine position. Research on 3D modelling of scoliotic spine began with multiplanar radiographs and later moved on to biplanar radiographs and finally a single radiograph. Nonetheless, modern advances in diagnostic imaging have the potential to preserve image quality and decrease radiation exposure. They include the DIERS formetric scanner system, the EOS imaging system, and ultrasonography. This review article briefly explains the technology behind each of these methods. They are compared with the standard imaging techniques. The DIERS system and ultrasonography are radiation free but have limitations with respect to the quality of the 3D model obtained. There is a need for 3D imaging technology with less or zero radiation exposure and that can produce a quality 3D model for diseases like adolescent idiopathic scoliosis. Accurate 3D models are crucial in clinical practice for diagnosis, planning surgery, patient follow-up examinations, biomechanical applications, and computer-assisted surgery.

Complete 3D Relationships Extraction Modality Alignment Network for 3D Dense Captioning.

3D dense captioning aims to semantically describe each object detected in a 3D scene, which plays a significant role in 3D scene understanding. Previous works lack a complete definition of 3D spatial relationships and the directly integrate visual and language modalities, thus ignoring the discrepancies between the two modalities. To address these issues, we propose a novel complete 3D relationship extraction modality alignment network, which consists of three steps: 3D object detection, complete 3D relationships extraction, and modality alignment caption. To comprehensively capture the 3D spatial relationship features, we define a complete set of 3D spatial relationships, including the local spatial relationship between objects and the global spatial relationship between each object and the entire scene. To this end, we propose a complete 3D relationships extraction module based on message passing and self-attention to mine multi-scale spatial relationship features and inspect the transformation to obtain features in different views. In addition, we propose the modality alignment caption module to fuse multi-scale relationship features and generate descriptions to bridge the semantic gap from the visual space to the language space with the prior information in the word embedding, and help generate improved descriptions for the 3D scene. Extensive experiments demonstrate that the proposed model outperforms the state-of-the-art methods on the ScanRefer and Nr3D datasets.

3D models for mohs micrographic surgery: a review on its use in patient education.

The use of a 3D model for patient education has shown encouraging results in surgical specialties like plastic surgery and neurosurgery, amongst many others; however, there is limited research on the clinical application of 3D models for Mohs Micrographic Surgery. This study delves into the utilization of 3D models for patient education in Mohs Surgery by juxtaposing different 3D modalities, highlighting their differences, and exploring potential avenues for future integration of 3D models into clinical practice. A literature search in the scientific database MEDLINE through PubMed and OVID and on the ProQuest Health & Medical Collection database was performed on the use of a 3D model for patient education. We limited the search to articles available in English and considered those mentioning the educational use of 3D models, especially for patient education, after excluding duplicate titles. We did not exclude articles based on publication year due to limited availability of literature. Utilizing 3D models for patient education within the framework of Mohs Micrographic surgery, including a 3D multicolored clay model and a 3D model accompanied by an educational video intervention, presents substantial advantages. 3D models offer a visual and tactile means to improve patients' comprehension of the Mohs procedure, the affected area, and possible outcomes. They hold the potential to reduce patient anxiety and improve decision-making. Currently, literature on the use of 3D models for patient education in Mohs Micrographic Surgery is limited, warranting further research in this area.

Complementary pseudo multimodal feature for point cloud anomaly detection

Point cloud anomaly detection is steadily emerging as a promising research area. Recognizing the importance of feature descriptiveness in this task, this study introduces the Complementary Pseudo Multimodal Feature (CPMF), which combines local geometrical information extracted by 3D handcrafted descriptors with global semantic information extracted from 2D pre-trained neural networks. Specifically, to leverage 2D pre-trained neural networks for point-wise feature extraction, this study projects original point clouds into multi-view images. These images are then fed into a pre-trained 2D neural network for informative 2D modality feature extraction. Following the 2D–3D correspondence, the multi-view 2D modality features are projected back to 3D space and aggregated to obtain point-wise 2D modality features. Finally, the point-wise 3D and 2D modality features are fused to derive the CPMF for point cloud anomaly detection. Extensive experiments conducted on MVTec 3D and Real3D datasets demonstrate the complementary capacity between 2D and 3D modality features and the effectiveness of CPMF. Notably, CPMF achieves a significantly higher object-level AUROC of 95.15% compared to other methods on the MVTec 3D benchmark. Code is available at https://github.com/caoyunkang/CPMF.

Quantitative 3D electron microscopy characterization of mitochondrial structure, mitophagy, and organelle interactions in murine atrial fibrillation

Atrial fibrillation (AF) is the most common clinical arrhythmia, however there is limited understanding of its pathophysiology including the cellular and ultrastructural changes rendered by the irregular rhythm, which limits pharmacological therapy development. Prior work has demonstrated the importance of reactive oxygen species (ROS) and mitochondrial dysfunction in the development of AF. Mitochondrial structure, interactions with other organelles such as sarcoplasmic reticulum (SR) and T-tubules (TT), and degradation of dysfunctional mitochondria via mitophagy are important processes to understand ultrastructural changes due to AF. However, most analysis of mitochondrial structure and interactome in AF has been limited to two-dimensional (2D) modalities such as transmission electron microscopy (EM), which does not fully visualize the morphological evolution of the mitochondria during mitophagy. Herein, we utilize focused ion beam-scanning electron microscopy (FIB-SEM) and perform reconstruction of three-dimensional (3D) EM from murine left atrial samples and measure the interactions of mitochondria with SR and TT. We developed a novel 3D quantitative analysis of FIB-SEM in a murine model of AF to quantify mitophagy stage, mitophagosome size in cardiomyocytes, and mitochondrial structural remodeling when compared with control mice. We show that in our murine model of spontaneous and continuous AF due to persistent late sodium current, left atrial cardiomyocytes have heterogenous mitochondria, with a significant number which are enlarged with increased elongation and structural complexity. Mitophagosomes in AF cardiomyocytes are located at Z-lines where they neighbor large, elongated mitochondria. Mitochondria in AF cardiomyocytes show increased organelle interaction, with 5X greater contact area with SR and are 4X as likely to interact with TT when compared to control. We show that mitophagy in AF cardiomyocytes involves 2.5X larger mitophagosomes that carry increased organelle contents. In conclusion, when oxidative stress overcomes compensatory mechanisms, mitophagy in AF faces a challenge of degrading bulky complex mitochondria, which may result in increased SR and TT contacts, perhaps allowing for mitochondrial Ca2+ maintenance and antioxidant production.

Complex aortic plaques: hidden danger in aortic stenosis. Role of transesophageal echocardiography.

Aortic stenosis is associated with aortic plaques in up to 85% of cases because they share risk factors and pathogenic pathways. Intrinsically, complex aortic plaques carry a high risk of stroke, which has also been demonstrated in the context of aortic stenosis, especially in patients who underwent percutaneous or surgical replacement. Transesophageal echocardiography (TEE) is the imaging test of choice to detect plaques in the thoracic aorta and classify them as complex plaques. Furthermore, the 3D modality allows us to better specify its dimensions and anatomical characteristics, such as added thrombi or the presence of ulcers inside. This review aims to evaluate the use of TEE to detect complex aortic plaques in patients with an indication for percutaneous or surgical aortic valve replacement. To highlight the association between aortic stenosis and complex aortic plaques, we attached to the review some TEE studies from our experience.

Intraoral ultrasonography image registration for evaluation of partial edentulous ridge: A methodology and validation study

ObjectivesUltrasound (US) reveals details for diagnosing soft- and hard-tissue dimensions around teeth, implants, and the edentulous ridge, not seen in 2D radiographs. Co-registering free-hand US scans with other 3D modalities presents reliability challenges. This study first aims to develop and validate a registration method to longitudinally reproduce US images of the jawbone on a simulator. In addition, it also evaluates the degree of the anatomical match in humans between US images acquired by the proposed registration method and the commonly used freehand acquisitions in comparison to cone beam computed tomography (CBCT) and intra-oral optical scan (IOS), used as references. MethodsA previously introduced ultrasound phantom was employed as a CBCT-US hybrid, suitable for training and technique development of US guides in edentulous ridges. After establishing feasibility in the phantom, the methodology was validated in a cohort of 24 human subjects (26 cases). Soft tissues were delineated on US and IOS, and hard tissues on US and CBCT. US accuracy and repeatability from both guided and freehand scans (non-guided) was assessed as the average distance between US and the references. ResultsGuided US images resembled the references more closely than freehand (non-guided) scans. Notably, delineation of soft and hard tissues was significantly more accurate when employing guides. In the phantom, guided scans exhibited an absolute mean deviation of 81.8 µm for gingiva and 90.4 µm for bone, whereas non-guided scans showed deviations of 150.4 µm and 177.2 µm, respectively. Similarly, in vivo, guided US outperformed non-guided US, with gingiva deviations of 125 µm and 196 µm, and bone deviations of 354 µm and 554 µm, respectively. ConclusionsBy using a registration method, guided US scans improved repeatability and accuracy of mapping hard and soft tissue of the edentulous ridge when compared to non-guided scans. Clinical relevanceThis guided US imaging method could lay the foundation for longitudinal evaluation of tissue behavior and dimensional changes with improved accuracy.

URCA: Uncertainty-based region clipping algorithm for semi-supervised medical image segmentation

Background and objectiveTraining convolutional neural networks based on large amount of labeled data has made great progress in the field of image segmentation. However, in medical image segmentation tasks, annotating the data is expensive and time-consuming because pixel-level annotation requires experts in the relevant field. Currently, the combination of consistent regularization and pseudo labeling-based semi-supervised methods has shown good performance in image segmentation. However, in the training process, a portion of low-confidence pseudo labels are generated by the model. And the semi-supervised segmentation method still has the problem of distribution bias between labeled and unlabeled data. The objective of this study is to address the challenges of semi-supervised learning and improve the segmentation accuracy of semi-supervised models on medical images. MethodsTo address these issues, we propose an Uncertainty-based Region Clipping Algorithm for semi-supervised medical image segmentation, which consists of two main modules. A module is introduced to compute the uncertainty of two sub-networks predictions with diversity using Monte Carlo Dropout, allowing the model to gradually learn from more reliable targets. To retain model diversity, we use different loss functions for different branches and use Non-Maximum Suppression in one of the branches. The other module is proposed to generate new samples by masking the low-confidence pixels in the original image based on uncertainty information. New samples are fed into the model to facilitate the model to generate pseudo labels with high confidence and enlarge the training data distribution. ResultsComprehensive experiments on the combination of two benchmarks ACDC and BraTS2019 show that our proposed model outperforms state-of-the-art methods in terms of Dice, HD95 and ASD. The results reach an average Dice score of 87.86 % and a HD95 score of 4.214 mm on ACDC dataset. For the brain tumor segmentation, the results reach an average Dice score of 84.79 % and a HD score of 10.13 mm. ConclusionsOur proposed method improves the accuracy of semi-supervised medical image segmentation. Extensive experiments on two public medical image datasets including 2D and 3D modalities demonstrate the superiority of our model. The code is available at: https://github.com/QuintinDong/URCA.

Radiographic diagnosis of periodontal diseases - Current evidence versus innovations.

Accurate diagnosis of periodontal and peri-implant diseases relies significantly on radiographic examination, especially for assessing alveolar bone levels, bone defect morphology, and bone quality. This narrative review aimed to comprehensively outline the current state-of-the-art in radiographic diagnosis of alveolar bone diseases, covering both two-dimensional (2D) and three-dimensional (3D) modalities. Additionally, this review explores recent technological advances in periodontal imaging diagnosis, focusing on their potential integration into clinical practice. Clinical probing and intraoral radiography, while crucial, encounter limitations in effectively assessing complex periodontal bone defects. Recognizing these challenges, 3D imaging modalities, such as cone beam computed tomography (CBCT), have been explored for a more comprehensive understanding of periodontal structures. The significance of the radiographic assessment approach is evidenced by its ability to offer an objective and standardized means of evaluating hard tissues, reducing variability associated with manual clinical measurements and contributing to a more precise diagnosis of periodontal health. However, clinicians should be aware of challenges related to CBCT imaging assessment, including beam-hardening artifacts generated by the high-density materials present in the field of view, which might affect image quality. Integration of digital technologies, such as artificial intelligence-based tools in intraoral radiography software, the enhances the diagnostic process. The overarching recommendation is a judicious combination of CBCT and digital intraoral radiography for enhanced periodontal bone assessment. Therefore, it is crucial for clinicians to weigh the benefits against the risks associated with higher radiation exposure on a case-by-case basis, prioritizing patient safety and treatment outcomes.

Three-dimensional measurement of humeral retroversion on a large academic cadaveric database

BackgroundThe Humeral retroversion angle (HRA) has been described in the literature as the orientation of the humeral head compared with the epicondylar axis of the distal humerus. HRA is a crucial measurement for designing shoulder prostheses and surgical technique, and is often noted to range from 25° to 35° in healthy adults. However, a wide range of individual variability has been reported in literature, with reported values ranging from −6° to 74°. Various imaging modalities including X-rays, computed tomography scans, and magnetic resonance imaging have historically been used to measure this angle, but conventional 2-dimensional technologies may result in inaccuracy and variability in angular measurements. Therefore, recent studies have focused on using 3-dimensional (3D) modalities to measure HRA. These studies have shown promising results regarding accuracy and clinical significance, although most have only included a small number of subjects and have not procured conclusive findings. This study aims to measure the HRA in a large sample of subjects using 3D imaging to establish measurements for the general population. MethodsWe examined the right and left cadaveric humerus from 559 individuals (146 females and 413 males). All of the humeri underwent computed tomography scan and surface models generated. 3D landmarks were automatically calculated on each 3D bone using custom-written software in C++. Those landmarks were used to calculate (1) HRA as the angle between the epicondylar axis and the humeral neck axis and (2) humeral proximal neck angle (HPNA) as the angle between the humeral neck axis and the anatomical axis. Descriptive statistics of both HRA and HPNA was analyzed using JMP Pro statistical software version 15.2.0. ResultsThe HPNA was found to be 137.7° ± 1.04° for males and 136.34° ± 1.4° for females with a 95% confidence interval. HRA was found to be 39.89° ± 12.77° for males and 38.89° ± 3.15° for females with a 95% confidence interval. Results of analysis of variance revealed that males had a statistically significant larger HRA than females (P < .001). ConclusionOur study suggests using a standardized measurement for the HRA, which we believe may improve operative outcomes. However, future prospective trials are required to validate our results in a clinical setting.

Sketch2Prototype: rapid conceptual design exploration and prototyping with generative AI

AbstractSketch2Prototype is an AI-based framework that transforms a hand-drawn sketch into a diverse set of 2D images and 3D prototypes through sketch-to-text, text-to-image, and image-to-3D stages. This framework, shown across various sketches, rapidly generates text, image, and 3D modalities for enhanced early-stage design exploration. We show that using text as an intermediate modality outperforms direct sketch-to-3D baselines for generating diverse and manufacturable 3D models. We find limitations in current image-to-3D techniques, while noting the value of the text modality for user-feedback.

Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA

In 3D Visual Question Answering (3D VQA), the scarcity of fully annotated data and limited visual content diversity hampers the generalization to novel scenes and 3D concepts (e.g., only around 800 scenes are utilized in ScanQA and SQA dataset). Current approaches resort supplement 3D reasoning with 2D information. However, these methods face challenges: either they use top-down 2D views that introduce overly complex and sometimes question-irrelevant visual clues, or they rely on globally aggregated scene/image-level representations from 2D VLMs, losing the fine-grained vision-language correlations. To overcome these limitations, our approach utilizes question-conditional 2D view selection procedure, pinpointing semantically relevant 2D inputs for crucial visual clues. We then integrate this 2D knowledge into the 3D-VQA system via a two-branch Transformer structure. This structure, featuring a Twin-Transformer design, compactly combines 2D and 3D modalities and captures fine-grained correlations between modalities, allowing them mutually augmenting each other. Integrating proposed mechanisms above, we present BridgeQA, that offers a fresh perspective on multi-modal transformer-based architectures for 3D-VQA. Experiments validate that BridgeQA achieves state-of-the-art on 3D-VQA datasets and significantly outperforms existing solutions. Code is available at https://github.com/matthewdm0816/BridgeQA.

MWSIS: Multimodal Weakly Supervised Instance Segmentation with 2D Box Annotations for Autonomous Driving

Instance segmentation is a fundamental research in computer vision, especially in autonomous driving. However, manual mask annotation for instance segmentation is quite time-consuming and costly. To address this problem, some prior works attempt to apply weakly supervised manner by exploring 2D or 3D boxes. However, no one has ever successfully segmented 2D and 3D instances simultaneously by only using 2D box annotations, which could further reduce the annotation cost by an order of magnitude. Thus, we propose a novel framework called Multimodal Weakly Supervised Instance Segmentation (MWSIS), which incorporates various fine-grained label correction modules for both 2D and 3D modalities, along with a new multimodal cross-supervision approach. In the 2D pseudo label generation branch, the Instance-based Pseudo Mask Generation (IPG) module utilizes predictions for self-supervised correction. Similarly, in the 3D pseudo label generation branch, the Spatial-based Pseudo Label Generation (SPG) module generates pseudo labels by incorporating the spatial prior information of the point cloud. To further refine the generated pseudo labels, the Point-based Voting Label Correction (PVC) module utilizes historical predictions for correction. Additionally, a Ring Segment-based Label Correction (RSC) module is proposed to refine the predictions by leveraging the depth prior information from the point cloud. Finally, the Consistency Sparse Cross-modal Supervision (CSCS) module reduces the inconsistency of multimodal predictions by response distillation. Particularly, transferring the 3D backbone to downstream tasks not only improves the performance of the 3D detectors, but also outperforms fully supervised instance segmentation with only 5% fully supervised annotations. On the Waymo dataset, the proposed framework demonstrates significant improvements over the baseline, especially achieving 2.59% mAP and 12.75% mAP increases for 2D and 3D instance segmentation tasks, respectively. The code is available at https://github.com/jiangxb98/mwsis-plugin.

Frozen CLIP Transformer Is an Efficient Point Cloud Encoder

The pretrain-finetune paradigm has achieved great success in NLP and 2D image fields because of the high-quality representation ability and transferability of their pretrained models. However, pretraining such a strong model is difficult in the 3D point cloud field due to the limited amount of point cloud sequences. This paper introduces Efficient Point Cloud Learning (EPCL), an effective and efficient point cloud learner for directly training high-quality point cloud models with a frozen CLIP transformer. Our EPCL connects the 2D and 3D modalities by semantically aligning the image features and point cloud features without paired 2D-3D data. Specifically, the input point cloud is divided into a series of local patches, which are converted to token embeddings by the designed point cloud tokenizer. These token embeddings are concatenated with a task token and fed into the frozen CLIP transformer to learn point cloud representation. The intuition is that the proposed point cloud tokenizer projects the input point cloud into a unified token space that is similar to the 2D images. Comprehensive experiments on 3D detection, semantic segmentation, classification and few-shot learning demonstrate that the CLIP transformer can serve as an efficient point cloud encoder and our method achieves promising performance on both indoor and outdoor benchmarks. In particular, performance gains brought by our EPCL are 19.7 AP50 on ScanNet V2 detection, 4.4 mIoU on S3DIS segmentation and 1.2 mIoU on SemanticKITTI segmentation compared to contemporary pretrained models. Code is available at \url{https://github.com/XiaoshuiHuang/EPCL}.

3D printing modality effect: Distinct printing outcomes dependent on selective laser sintering (SLS) and melt extrusion.

A direct and comprehensive comparative study on different 3D printing modalities was performed. We employed two representative 3D printing modalities, laser- and extrusion-based, which are currently used to produce patient-specific medical implants for clinical translation, to assess how these two different 3D printing modalities affect printing outcomes. The same solid and porous constructs were created from the same biomaterial, a blend of 96% poly-ε-caprolactone (PCL) and 4% hydroxyapatite (HA), using two different 3D printing modalities. Constructs were analyzed to assess their printing characteristics, including morphological, mechanical, and biological properties. We also performed an in vitro accelerated degradation study to compare their degradation behaviors. Despite the same input material, the 3D constructs created from different 3D printing modalities showed distinct differences in morphology, surface roughness and internal void fraction, which resulted in different mechanical properties and cell responses. In addition, the constructs exhibited different degradation rates depending on the 3D printing modalities. Given that each 3D printing modality has inherent characteristics that impact printing outcomes and ultimately implant performance, understanding the characteristics is crucial in selecting the 3D printing modality to create reliable biomedical implants.

The coronal alignment differs between two-dimensional weight-bearing and three-dimensional nonweight bearing planning in total knee arthroplasty.

The goal of this study is (1) to assess differences between two-dimensional (2D) weight-bearing (WB) and three-dimensional (3D) nonweight-bearing (NWB) planning in total knee arthroplasty (TKA) and (2) to identify factors that influence intermodal differences. Retrospective single-centre analysis of patients planned for a TKA with patient-specific instruments (PSI). Preoperative WB long-leg radiographs and NWB computed tomography were analysed and following radiographic parameters included: hip-knee-ankle angle (HKA) (+varus/-valgus), joint line convergence angle (JLCA), femorotibial subluxation and bony defect classified according to Anderson. Preoperative range of motion was also considered as possible covariate. Demographic factors included age, sex, and body mass index. A total of 352 knees of 323 patients (66% females) with a mean age of 66 ± 9.7 years were analysed. The HKA differed significantly between 2D and 3D planning modalities; varus knees (n = 231): 9.9° ± 5.1° vs. 6.7° ± 4°, p < 0.001; valgus knees (n = 121): -8.2° ± 6° vs. -5.5° ± 4.4°, p < 0.001. In varus knees, HKA (β = 0.38; p < 0.0001) and JLCA (β = 0.14; p = 0.03) were associated with increasing difference between 2D/3D HKA. For valgus knees, HKA (β = -0.6; p < 0.0001), JLCA (β = -0.3; p = 0.0001) and lateral distal femoral angle (β = -0.28; p = 0.03) showed a significant influence on the mean absolute difference. The coronal alignment in preoperative 3D model for PSI-TKA significantly differed from 2D WB state and the difference between modalities correlated with the extent of varus/valgus deformity. In the vast majority of cases, the 3D NWB approach significantly underestimated the preoperative deformity, which needs to be considered to achieve the planned correction when using PSI in TKA. Level III.

CMFAN: Cross-Modal Feature Alignment Network for Few-Shot Single-View 3D Reconstruction.

Few-shot single-view 3D reconstruction learns to reconstruct the novel category objects based on a query image and a few support shapes. However, since the query image and the support shapes are of different modalities, there is an inherent feature misalignment problem damaging the reconstruction. Previous works in the literature do not consider this problem. To this end, we propose the cross-modal feature alignment network (CMFAN) with two novel techniques. One is a strategy for model pretraining, namely, cross-modal contrastive learning (CMCL), here the 2D images and 3D shapes of the same objects compose the positives, and those from different objects form the negatives. With CMCL, the model learns to embed the 2D and 3D modalities of the same object into a tight area in the feature space and push away those from different objects, thus effectively aligning the global cross-modal features. The other is cross-modal feature fusion (CMFF), which further aligns and fuses the local features. Specifically, it first re-represents the local features with the cross-attention operation, making the local features share more information. Then, CMFF generates a descriptor for the support features and attaches it to each local feature vector of the query image with dense concatenation. Moreover, CMFF can be applied to multilevel local features and brings further advantages. We conduct extensive experiments to evaluate the effectiveness of our designs, and CMFAN sets new state-of-the-art performance in all of the 1-/10-/25-shot tasks of ShapeNet and ModelNet datasets.