Does colours and patterns affect your branding? Yes!
It is important to have some kind of theme and colour scheme across your feeds, mostly if Instagram is part of your social strategy.
When you are starting out, building your community and establishing your brand, having a brand theme is one of the key elements to help you get an identity. Having an identity, builds trust.
If you constantly change up your style, colours and visuals, it sounds like you don’t what you stand for. Your visuals need to make sense and reflect who you are as a business.