In some cases, you might find that some objects are unexpectedly selected (and moved) while you're taking notes.
It might be due to a feature called Universal Object Selection, which allows selecting non-ink objects with a finger-tap while any editing tool is active.
If the feature disrupts your workflow, you can turn it off: Open More menu (...) in the top-right corner > Document Editing, then disable "Image and text box".
See also: Select content for copying, moving, etc.