Best I can tell from quick internet searches: Old English: wīfmann/menn (“female person/s”). The w rounded the following vowel giving a wo- pronunciation, which for some reason (umlaut?) stuck for the singular but not the plural. The spelling of the plural changed to match that of the singular in spite of the pronunciation.

* Everything here carries the caveat “in some dialects, …” because English