In the Motion Dock, enter Edit layout and double-click Roundhouse’s slot. You can also right-click its slot and choose Module settings. Leave layout mode after closing settings to use Roundhouse again.

These settings only decide how new corners start. They never change corners you already rounded.

SettingDefaultOptionsBehavior
New Corner StyleRoundRound, Chamfer, Notch, SquareThe style a sharp corner takes the first time you drag it.
New Corner Smoothing0% · circular0% · circular, 60% · iOS, 100% · smoothestThe Smoothing a shape or layer starts with the first time you round it.
Drag Step1 px0.5 px, 1 px, 5 px, 10 pxHow the radius steps while you drag a corner. A drag all the way in always lands on the largest radius.