Can prolonged use of nicotine lozenges cause stomach ulcers or does the nicotine aggravate ulcers?

This is from the FDA Nicotine monograph on side-effects:

"Gastrointestinal side effects have included nausea, dry mouth, dyspepsia, and diarrhea with the patch and gum formulations in approximately 6% of patients. Use of nicotine gum may also cause hiccups, flatulence, increased salivation, stomatitis, tooth disorder, glossitis, and unpleasant taste. The use of nicotine has been shown to decrease lower esophageal sphincter pressure. Heartburn has been associated with nicotine lozenges in 5% to 5.8% of patients."

Yes, the Nicotine lozenges could be aggravating a stomach ulcer. You should discuss your symptoms with your doctor.
For a full and accurate list of Nicotine side-effects see: https://www.drugs.com/sfx/nicotine-side-effects.html