Payments & Invoicing aren't supported with custom objects. For family and team memberships, the subscription belongs to the account (a custom object record), not just the payer's contact. Please allow associating a subscription/invoice with a custom object record, and workflow triggers like "Subscription payment failed" that can update the associated record.