

5e RAW, they remain invisible. However, using the tinderbox will make noise so enemies will likely notice that.
In the 2024 rules, dealing damage (even if not by attacking) causes invisibility to end.
NOTE: both versions of the rules (see the oil item description) state that in this scenario, an enemy only takes damage if it enters or ends its turn in the oiled space. Starting their turn in the burning oil does not cause damage, so just moving out of the space will prevent the damage. The lighting of the oil also does not cause immediate damage
The 2024 spell ends immediately if you deal any damage. So, also item damage.