Part of updating the website included changing the key vault permission from Vault Access Policy to the recommended role-based access control.

This required assigning the correct role to the new web app. Neither Reader or Key Vault Reader had sufficient privilege to access the secret values. Key Vault Secret User appears to be the correct choice.

I also had to assign the Key Vault Administrator role to my administrative account in order to access the individual key vault secrets through the Azure interface.

Checking the Identity section of the web app confirms the correct role has been assigned.