You may still have legal recourse against the individual caregiver for their acts of abuse, neglect, or theft. However, without an agency involved, there may not be the same depth of insurance coverage available. An attorney can evaluate the specifics of your situation.