Pathology　imaging　is　broadly　used　for　identifying　the　causes　and　effects　of　diseases　or　injuries.　Given　a　pathology　image,　being　able　to　answer　questions　about　the　clinical　findings　contained　in　the　image　is　very　important　for　medical　decision　making.　In　this　paper,　we　aim　to　develop　a　pathological　visual　question　answering　framework　to　analyze　pathology　images　and　answer　medical　questions　related　to　these　images.　To　build　such　a　framework,　we　create　PathVQA,　a　pathology　VQA　dataset　with　32,795　questions　asked　from　4,998　pathology　images.　We　also　propose　a　three-level　optimization　framework　which　performs　self-supervised　pretraining　and　VQA　finetuning　end-to-end　to　learn　powerful　visual　and　textual　representations　jointly　and　automatically　identifies　and　excludes　noisy　self-supervised　examples　from　pretraining.　We　perform　experiments　on　our　created　PathVQA　dataset　and　the　results　demonstrate　the　effectiveness　of　our　proposed　methods.　The　datasets　and　code　are　available　at　https://github.com/UCSD-AI4H/PathVQA