Histone deacetylase (HDAC) family proteins play important roles in pathological alterations of Alzheimer’s disease (AD). There has been keen interest in developing small molecule inhibitors of HDACs as potential therapeutics of AD. Presently, little is known about the role of HDAC11 in AD and whether HDAC11 can be a potential drug target for AD. Recently, we discovered PB94 as our lead small molecule inhibitor of HDAC11 exhibiting strong potency and selectivity against HDAC11 over other HDAC isoforms, which warranted further optimization and investigation for AD. Here, we optimized PB94 and analyzed the effects of PB94 and its analogs on amyloid 5xFAD mice and tau P301S mice. Our optimization of PB94 generated a viable library of PB94 analogs, including PB151 which displayed strong potency and selectivity toward HDAC11 as well as desirable safety profiles. Additionally, we demonstrated that pharmacological inhibition of HDAC11 led to reduced amyloid plaques or tau pathology and neuroinflammation by neuroimaging and improved cognition in these animals. Regarding mechanisms, we found that PB94 led to anti-AD effects through reducing microglia-related neuroinflammation by PET imaging in 5xFAD animals. We also utilized a mouse microglia BV2 cell model and found that PB94 increased microglial Aβ phagocytosis. Collectively, our findings show that HDAC11 may be a potential target for AD drug development and our results are setting the stage for further development of PB94 and its analogs as potential therapeutics against AD.