I guess I shouldn't have maintained my app store listing late at night, while everyone in my family was asleep. Late at night people are sleepy and when the're sleepy they make mistakes.
So somehow I ended up with a typo in my app's title in the store listing. I listed a "Baby" app instead of a "Babi Monitor" app, just missed the second word. Probably a copy-&-paste issue. The impact is pretty obvious:
While downloads did not completely vanish a decrease is clearly visible. Lesson learned: do your homework and avoid typos.