You can make use of the default VB.NET library functions such as My.Computer.FileSystem.DeleteFile (look it up on MSDN for the proper documentation, I cannot post links yet due to me being a new member), and these functions will redirect to the Windows API in the background anyway.
If you want to remove files in a more secure manner then you can use methods which are a bit more advanced than the above such as: over-writing the data within the target file with a bunch of random data for at least 5 times (Gutmann Method would be passing over the file 35 times) before actually deleting it; and using a function exported by ntdll.dll called NtDeleteFile which will allow you to delete a file even if it is in use by another process (you can P/Invoke the function from ntdll.dll).
The first method I mentioned above would be beneficial to prevent the recovery of the files data after removal, and the second method is good for targeting files which may already be in use by another process - bear in mind that when you use the default functions from the .NET libraries it becomes redirected to the Win32 API functions and then it gets redirected to the NTAPI (Native API) if the execution for the function call is allowed to continue, although the Win32 API functions do more check-ups which means if you call DeleteFile (exported by kernel32.dll which is where the My.Computer.FileSystem.DeleteFile will cause execution flow to end up) on a file being used by a process (e.g. opened by notepad.exe) then the removal will not succeed. Although if you call ntdll.dll!NtDeleteFile directly via P/Invoke (or a system call which is what ntdll.dll will do) then you can still remove files in use by other programs.
Good luck. 🙂
