Creating a Database Other Applications Can Use ?

I am writing an application that I wish to store data in a database.
Recently coming over across from MS Windows and being so used to
creating apps that have access to a database file that one can actually
see - lets say it's called 'MyData.mdb' then writing code to access this
database.

Under Linux and PostgreSQL I expected to create a single database file
that I then would write code to access, but all I can see is a directory
that contains a couple more directories that have numbers as their names
and then contain many files that again are named by numeric values.

Do I still need to go that one step further and create a single database
file that my code would use ?

I heard that pg_dump may have created a file but again I can't seem to
find this file anywhere..... what am I doing wrong ?

Pete