You can manage who has read/write access to the Diary from the Diary as well as from the Supported User's Circle page. See 'Managing Access from the Circle' to manage Diary access from here. To manage Diary Access from the Diary please follow the following steps:
NOTES:
- You will not be able to 'Edit Access' of Diary Admins, they will have access to everything. You have to turn them off as Admins first in the 'Is Admin' column before you can edit their access
- You cannot give email access link users edit access to the Blog, they will need to register an account first to be able to edit the Blog section of the diary