On reflection, what you say is true. I guess I'm trying to understand why this was done in Latin, which I have always viewed as a rule-based, rule-governed language, as opposed to English, wherein rules are made to be broken.
The sooner you stop believing in that myth the better. There's actually a fair bit of chaos in Latin too. Incidentally, you may amuse yourself reading about what is actually attested in case endings. There's always been some ambiguity/variation on what the difference between the ablative singular endings -ī and -e is exactly, even if you restrict yourself to the classical era, and quite a number of weird exceptions (ignis 'fire' is commonly either ignī or igne but otherwise has genitive plural ignium and the optional i-stem accusative ignīs, see also the adjective vetus 'old' and its noun-y declension with vetere vetera veterum, and -āns active participles are a mess).
A seemingly obvious question: is there a term in linguistics for such "borrowing" within inflected languages, of parts of speech in a particular case for use as different parts of speech, as is represented by adjective (neuter accusative) > adverb (iūge), or adjective (feminine accusative) > adverb (perperam)?
Yes, it's called "zero-derivation" ("zero" here means that nothing has been altered in the base). Stereotypically the term is used for non-inflected languages (Mandarin 危險 wēixiǎn 'dangerous' > 'danger'), but it can also apply to inflected ones.
Alternatively there is also "fossilization", especially if it's an expression (a fossilized phrase), e.g. Spanish
correveidile 'tattler, ratter, telltale', which is literally
corre, ve y dile 'run, see and tell him/her'. It takes the plural
los correveidiles. Fossilization is better for single words when the ending is used in an unusual way though (as in
partim 'partially').