The human immunodeficiency virus (HIV) accessory protein Nef plays a major role in establishing and maintaining infection, particularly through immune evasion. Many HIV-2-infected people experience long-term viral control and survival, resembling HIV-1 elite control. HIV-2 Nef has overlapping but also distinct functions from HIV-1 Nef. Here we report the crystal structure of HIV-2 Nef core. The di-leucine sorting motif forms a helix bound to neighboring molecules, and moreover, isothermal titration calorimetry demonstrated that the CD3 endocytosis motif can directly bind to HIV-2 Nef, ensuring AP-2-mediated endocytosis for CD3. The highly conserved C-terminal region forms a α-helix, absent from HIV-1. We further determined the structure of simian immunodeficiency virus (SIV) Nef harboring this region, demonstrating similar C-terminal α-helix, which may contribute to AP-1 binding for MHC-I downregulation. These results provide insights into the distinct pathogenesis of HIV-2 infection.