Head　pose　estimation　(HPE)　is　an　indispensable　upstream　task　in　the　fields　of　human-machine　interaction,　self-driving,　and　attention　detection.　However,　practical　head　pose　applications　suffer　from　several　challenges,　such　as　severe　occlusion,　low　illumination,　and　extreme　orientations.　To　address　these　challenges,　we　identify　three　cues　from　head　images,　namely,　critical　minority　relationships,　neighborhood　orientation　relationships,　and　significant　facial　changes.　On　the　basis　of　the　three　cues,　two　key　insights　on　head　poses　are　revealed:　1)　intra-orientation　relationship　and　2)　cross-orientation　relationship.　To　leverage　two　key　insights　above,　a　novel　relationship-driven　method　is　proposed　based　on　the　Transformer　architecture,　in　which　facial　and　orientation　relationships　can　be　learned.　Specifically,　we　design　several　orientation　tokens　to　explicitly　encode　basic　orientation　regions.　Besides,　a　novel　token　guide　multi-loss　function　is　accordingly　designed　to　guide　the　orientation　tokens　as　they　learn　the　desired　regional　similarities　and　relationships.　Experimental　results　on　three　challenging　benchmark　HPE　datasets　show　that　our　proposed　TokenHPE　achieves　state-of-the-art　performance.　Moreover,　qualitative　visualizations　are　provided　to　verify　the　effectiveness　of　the　token-learning　methodology.