An ARIA role is a promise, and a broken one is worse than none
Putting role="radiogroup" on a loop of buttons tells a screen reader that arrow keys move between the options. If you have not implemented that, you have made the control harder to use than plain buttons would have been.
Do I need keyboard handling if I add an ARIA role?
Yes. role="radiogroup" tells assistive technology that the group is a single tab stop and that arrow keys move the selection, so adding the role without that behaviour leaves the user worse off than plain buttons, which at least behave the way they are announced. Either implement the keyboard pattern — one tab stop, arrows on both axes, Home and End — or drop the role and use buttons with aria-pressed, which is a smaller promise you are already keeping.