Is Dream Hunter Rem hentai?
No
Rated R+ (Mild Nudity) on MyAnimeList and not tagged as Hentai.

OVA · 1985 · 4 episodes · R+ - Mild Nudity
Action, Ecchi, Mahou Shoujo
In a shabby building hidden in the back alleys of Tokyo, there is a detective's office that will take any case. Run by the youthful Rem Ayanokouji, the office specializes in bizarre cases where dreams and demons intersect. With the help of her two animal companions, Alpha and Beta, Rem uses her powers to enter her clients' dreams to confront their issues—as well as the demons tormenting them—directly. [Written by MAL Rewrite]
View on MyAnimeListShare
Check another title