In this paper, we present a PET and MR brain image fusion method based on wavelet transform for low- and high-activity brain image regions, respectively. Our method can generate very good fusion result by adjusting the anatomical structural information in the gray matter (GM) area, and then patching the spectral information in the white matter (WM) area after the wavelet decomposition and gray-level fusion. We used normal axial, normal coronal, and Alzheimer’s disease brain images as the three datasets for testing and comparison. Experimental results showed that the performance of our fusion method is better than that of IHS+RIM fusion method in terms of spectral discrepancy (SD) and average gradient (AG). In fact, our method is superior to IHS+RIM method both visually and quantitatively.