This just requires a bit of bookkeeping that keeps track of the task ID to the download manager ID.
Note that the behavior of the download manager remove method is to remove the download and the downloaded file, whether partial or complete.
https://developer.android.com/reference/android/app/DownloadManager.html#remove(long...)