The answer to this question strongly depends on your area of expertise. As a rule of thumb, professions in which you could potentially harm people (doctors, teachers, engineers, lawyers, etc.) need to seek accreditation of their diplomas. Usually these jobs are within a list of regulated professions
. If you are an Economist or a specialist in IT, you may find a job without the official accreditation. You will still need to have documents translated, though. You will find a complete guide
to diploma accreditation on the official site of the Ministry of Education and Research (BMBF).