Is it rude to say “It doesn’t matter” when I actually mean “I don’t mind”?

Hi everyone! I'm confused about these two phrases. If a friend asks me, "Do you want to eat Italian or Mexican food tonight?", which one is more natural? "I don't mind." "It doesn't matter." To me, they both feel like I'm saying "both are okay," but I've heard that "It doesn't matter" can sometimes sound a bit cold or like I don't care about the plans at all. Is "I don't mind" considered more polite or "warmer" in social situations? I'd love to hear some examples of when you would use one over the other!