For security reasons, it's often a good idea to have:
present in your /etc/my.cnf file.
If you want to set all users except da_admin and root to not have that privilege at the mysql level, then login to mysql with the login/pass from /usr/local/directadmin/conf/mysql.conf, and run the query:
UPDATE mysql.user SET File_priv='N' WHERE user!='da_admin' AND user!='root';
**NOTE** that that above command will remove file access from *all* accounts on your system, except da_admin and root.. so just be aware that it's a global change.