This　paper　investigated　the　rheological　properties　of　asphalt　binder,　asphalt　mastic　and　asphalt　mortar　and　the　interaction　between　asphalt　binder,　mineral　filler　and　fine　aggregates.　Asphalt　binder,　mastic　and　mortar　can　be　regarded　as　the　binding　phase　at　different　scales　in　asphalt　concrete.　Asphalt　mastic　is　a　blend　of　asphalt　binder　and　mineral　filler　smaller　than　0.075　mm　while　asphalt　mortar　consists　of　asphalt　binder,　mineral　filler　and　fine　aggregate　smaller　than　2.36　mm.　The　material　compositions　of　mastic　and　mortar　were　determined　from　the　commonly　used　asphalt　mixtures.　Dynamic　shear　rheometer　was　used　to　conduct　rheological　analysis　on　asphalt　binder,　mastic　and　mortar.　The　obtained　test　data　on　complex　modulus　and　phase　angle　were　used　for　the　construction　of　rheological　master　curves　and　the　investigation　of　asphalt-filler/aggregate　interaction.　Test　results　indicated　a　modulus　increase　of　three-　to　five-fold　with　the　addition　of　filler　and　a　further　increase　of　one　to　two　orders　of　magnitude　with　cumulative　addition　of　fine　aggregates　into　asphalt　binder.　Fine　aggregates　resulted　in　a　phase　change　for　mortar　at　high　temperatures　and　low　frequencies.　The　filler　had　stronger　physical　interaction　than　fine　aggregate　with　an　interaction　parameter　of　1.8-2.8　and　1.15-1.35　respectively.　Specific　area　could　enhance　asphalt-filler　interaction.　The　mastic　and　mortar　modulus　can　be　well　predicted　based　on　asphalt　binder　modulus　by　using　particle　filling　effect.　Asphalt　mortar　had　a　significant　modulus　reinforcement　and　phase　change　and　thus　could　be　the　closest　subscale　in　terms　of　performance　to　that　of　asphalt　mixtures.　It　could　be　a　vital　scale　that　bridges　the　gap　between　asphalt　binder　and　asphalt　mixtures　in　multiscale　performance　analysis.