About what can and can't be considered appropriation:
Besides, what makes the naysayers voices more important than the ones who encourage it?
Because cultural appropriation hurts the people who speak out against it, while the naysayers claiming that everything is to be shared are basically unaffected
Instead of promoting segregation, we should be promoting cultural exchange and knowledge so we can slowly correct some of the negative stereotypes.
Not everything relating to culture is to be shared. Not every part of a culture is up for the consumption of outsiders. That's not segregation, that's cultural preservation in an Imperialist and Colonialist society that simultaneously destroys and monetizes cultures.
And who gets to decide that? Who gets to decide what gets to be shared and what is "too sacred" to share? By you're own reasoning, nothing can be shared ever because if even
one person complains,
their opinion is the one that's held as absolutely sacred while the voices of the other people of the same culture are silenced and trampled because "they don't matter unless they're hurting enough".