Drugs that modify the risk of dementia in the elderly are of potential interest for dementia prevention. Proton pump inhibitors (PPIs) are widely used to reduce gastric acid production, but information on the risk of dementia is lacking. We assessed association between the use of PPIs and the risk of dementia in elderly people. Data were derived from a longitudinal, multicenter cohort study in elderly primary care patients, the German Study on Aging, Cognition and Dementia in Primary Care Patients (AgeCoDe), including 3,327 community-dwelling persons aged ≥75 years. From follow-up 1 to follow-up 4 (follow-up interval 18 months), we identified a total of 431 patients with incident any dementia, including 260 patients with Alzheimer’s disease. We used time-dependent Cox regression to estimate hazard ratios of incident any dementia and Alzheimer’s disease. Potential confounders included in the analysis comprised age, sex, education, the Apolipoprotein E4 (ApoE4) allele status, polypharmacy, and the comorbidities depression, diabetes, ischemic heart disease, and stroke. Patients receiving PPI medication had a significantly increased risk of any dementia [Hazard ratio (HR) 1.38, 95 % confidence interval (CI) 1.04–1.83] and Alzheimer’s disease (HR 1.44, 95 % CI 1.01–2.06) compared with nonusers. Due to the major burden of dementia on public health and the lack of curative medication, this finding is of high interest to research on dementia and provides indication for dementia prevention.