Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the causal agent for coronavirus disease 2019 (COVID-19). Although vaccines have helped to prevent uncontrolled viral spreading, our understanding of the fundamental biology of SARS-CoV-2 infection remains insufficient, which hinders effective therapeutic development. Here, we found that Apolipoprotein E (ApoE), a lipid binding protein, is hijacked by SARS-CoV-2 for infection. Preincubation of SARS-CoV-2 with a neutralizing antibody specific to ApoE led to inhibition of SARS-CoV-2 infection. The ApoE neutralizing antibody efficiently blocked SARS-CoV-2 infection of human iPSC-derived astrocytes and air-liquid interface organoid models in addition to human ACE2-expressing HEK293T cells and Calu-3 lung cells. ApoE mediates SARS-CoV-2 entry through binding to its cellular receptors such as the low density lipoprotein receptor (LDLR). LDLR knockout or ApoE mutations at the receptor binding domain or an ApoE mimetic peptide reduced SARS-CoV-2 infection. Furthermore, we detected strong membrane LDLR expression on SARS-CoV-2 Spike-positive cells in human lung tissues, whereas no or low ACE2 expression was detected. This study provides a new paradigm for SARS-CoV-2 cellular entry through binding of ApoE on the lipoviral particles to host cell receptor(s). Moreover, this study suggests that ApoE neutralizing antibodies are promising antiviral therapies for COVID-19 by blocking entry of both parental virus and variants of concern.