Are SQL commands "atomic" ?

I'm using 7.1.1 right now, and have the following table:

id | s
----+-------
1 | alpha
2 | beta
3 | gamma
4 | delta
(4 rows)

I'd like to switch the id of "beta" to 3 and the id of "gamma" to 2 ("flip"
them). Since id is the PK, it must remain unique and so I can't just set
the two lines using two UPDATEs.

My solution is:

UPDATE t1 SET id=id#1 WHERE id=2 OR id=3; -- # is the XOR operator

where 2#1=3 and 3#1=2. One statement will change both values as I want. But
when I run the statement, the server replies with:

ERROR: Cannot insert a duplicate key into unique index t1_pkey

If the statement is "atomic", then if the statement succeeds, the IDs will
be unique and the error is incorrect. Does this imply that SQL statements
are not actually atomic?