Portfolio Lending and Mortgage Access Act (Sec. 2) This bill amends the Truth in Lending Act to allow a depository institution or credit union with assets below a specified threshold to forgo certain ability-to-pay requirements regarding residential mortgage loans. Specifically, those requirements are waived if a loan: (1) is originated by and continuously retained by the institution, (2) complies with requirements regarding prepayment penalties and points and fees, and (3) does not have negative amortization or interest-only terms. Furthermore, for such requirements to be waived, the institution must consider and verify the debt, income, and financial resources of the consumer. The bill also provides for circumstances in which such requirements shall be waived with respect to a loan that is transferred: (1) by reason of bankruptcy or failure of the originating institution, (2) to a similar institution, (3) in the event of a merger, or (4) to a wholly owned subsidiary of the institution.