Because the NSF sometimes releases opportunities that deviate from their stated policy on effort conferred to Key Persons, they have given S2S users a workaround for changing proposals after submission. Here are the instructions for removing a PI from a proposal in FastLane after submitting via Cayuse Proposals, note you will need to have a FastLane login.
After you have successfully submitted with Cayuse go to FastLane and click "Proposals, Awards and Status"
Login to FastLane
Click on Proposal Functions...
Click on Revise Submitted Proposal Budget.
Find your study in the list, select it, and click View.
Select Edit the Budget and click OK.
You will get a screen where you can select a category to edit in, select Personnel.
You will get a list of Personnel, check the one to remove (the PI) and click Save.
Viewing the budget now, you should see no entry for the PI.