Voluntary Grazing Permit Retirement Act This bill authorizes the voluntary waiver of permits or leases for grazing on federal lands managed by the Department of Agriculture or the Department of the Interior in the 16 western states (e.g., Utah, Washington, and Wyoming). If a permit or lease is waived by a permittee or lessee with the intention of permanently ending livestock grazing, the appropriate department must accept and terminate, on a first-come, first-served basis, the permit or lease; refrain from issuing any new grazing permit or lease within the grazing allotment covered by the permit or lease; and ensure a permanent end to livestock grazing on the allotment covered by the permit or lease. If an allotment covered by a waiver is also covered by another permit or lease that is not waived, the department must reduce the level of commercial livestock grazing on the allotment to reflect the waiver. The departments must not accept more than (1) 100 grazing permits per year, in the aggregate for all of the 16 western states; and (2) 25 grazing permits for land located in whole or in part in any individual state.