Fragile X syndrome (FXS) patients present neuronal alterations that lead to severe intellectual disability, but the underlying neuronal circuit mechanisms are poorly understood. An emerging hypothesis postulates that reduced GABAergic inhibition of excitatory neurons is a key component in the pathophysiology of FXS. Here, we directly test this idea in a FXS Drosophila model. We show that FXS flies exhibit strongly impaired olfactory behaviors. In line with this, olfactory representations are less odor specific due to broader response tuning of excitatory projection neurons. We find that impaired inhibitory interactions underlie reduced specificity in olfactory computations. Finally, we show that defective lateral inhibition across projection neurons is caused by weaker inhibition from GABAergic interneurons. We provide direct evidence that deficient inhibition impairs sensory computations and behavior in an in vivo model of FXS. Together with evidence of impaired inhibition in autism and Rett syndrome, these findings suggest a potentially general mechanism for intellectual disability.