Special Drawing Rights Oversight Act of 2023 This bill imposes additional restrictions on the U.S. government's authority to vote on Special Drawing Rights (SDR) allocations at the International Monetary Fund (IMF). (The SDR is an international reserve asset maintained by the IMF based on contributions from IMF member countries. SDRs may be exchanged between member countries and may also be exchanged for currencies.) Under current law, U.S. representatives to the IMF may not vote for SDR allocations to the United States beyond an amount authorized by statute unless Congress authorizes such a vote. This bill further reduces the allocation amount that U.S. representatives to the IMF may vote for without congressional approval. Furthermore, U.S. representatives to the IMF may not vote for SDR allocations to a country if the President finds that the country's government has (1) committed genocide in the last 10 years, or (2) repeatedly supported international terrorism.