This study uses the cognitive factor of “visual harmony” to assess the visual quality of stream engineering in a mountainous region. Images of engineering structures such as revetments and submerged dams in the mountain streams of Taiwan were collected. Three image groups with different structures invaded by vegetation were used for a questionnaire survey, which yielded 154 valid samples. We used statistical analysis to develop a model of visual harmony H with respect to the percentage of visible greenery GR, that is, the perceived curve of vegetation change. A comparison of our data with the literature determined the upper and lower bound curves of the relationship between H and GR . We found that the physical elements of “softscape” and “hardscape”—namely, percentage of visible water WR, visible structure IR, and visible natural material on the structure NR —affected this relationship. Results show that H is equivalent to visual preference P, and both can be improved by better green visibility (increasing GR and GR < 50%), avoiding low water visibility (WR < 10%), or increasing the amount of visible natural material (NR > 0.9). High visibility of the structures (IR > 0.3) may decrease H and P. We ultimately propose a visual harmony or preference model concerning a combined physical indicator that comprises GR, WR, IR and NR . Results of this study could be helpful to improve or access the aesthetics of stream engineering design. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.