"Problems"? These things will run in open loop (what happens when you remove the O2 sensor) however it's not how they were designed to run. Using the O2 sensor helps clean up emissions (IE, less stink from the exhaust), and will almost certainly net you better MPG.
Assuming this is TBI, trying to make it less complex is like going back to a carburetor. Normally better to just let it be unless you like making problems for yourself.
Headers alone will cause a lean condition that requires a chip (PROM) tune to fix, but that's another issue.