One reason to be less nervous when giving a speech is that

there are people in the audience who are even worse at public speaking than you

your topic probably isn't important enough to your audience for them to care

much, if not most, of your nervousness goes undetected by the audience

nervousness is never a good thing because it is an indication that you don't care about delivering a quality speech

nervousness suggests that you have little idea what you are talking about