Restoring Trade Fairness Act This bill establishes various trade measures related to China, including by revoking China's permanent normal trade relations (PNTR) status and increasing the rates of duty (i.e., tariffs) on Chinese imported goods. The bill prohibits imported goods originating from North Korea, China, Russia, or Iran from receiving de minimis treatment. (Current law allows for U.S. imports under a de minimis threshold of $800 per shipment to enter free of tariffs, fees, and taxes.) Specifically, the bill revokes China's PNTR status. Currently, China's PNTR status allows for Chinese goods to have duty rates set forth in column 1 of the Harmonized Tariff Schedule of the United States (HTS). With the removal of China's PNTR status, the bill generally sets the applicable duty rates on imported Chinese goods at the higher rates listed in column 2 of the HTS, with exceptions. The bill establishes a minimum duty rate of 35% for all Chinese goods, which requires column 2 rates to be at least 35%. However, the bill establishes a minimum duty rate of 100% for a list of specified goods (e.g., various minerals, certain vaccines and drugs, and certain defense-related articles). Duty rates are phased in over five years and adjusted annually for inflation. The bill also authorizes the President to take additional actions related to trade with China, requires merchandise imported from China to be appraised based on U.S. value, and establishes a trust fund to compensate U.S. producers for lost revenue resulting from retaliatory actions by China.