I had a suspicion and it was confirmed:
test=# create table oidtest (a int4, unique(oid));
NOTICE: CREATE TABLE / UNIQUE will create implicit index
"oidtest_oid_key" for table "oidtest"
CREATE TABLE
test=# select oid from oidtest;
oid
-----
(0 rows)
test=# alter table oidtest set without oids;
ALTER TABLE
test=# select oid from oidtest;
ERROR: column "oid" does not exist
test=# \d oidtest
ERROR: cache lookup failed for attribute -2 of relation 765798
Chris