RPE 6 work for someone like me would be quite light. For example my best bench ever is 205 and right now I doubt I could go bench 185. 4 reps in the tank on my bench might be like 120lbs only is that going to be ok?
Yes, absolutely it is ok. RPE is easily scale-able and will be applicable regardless of your strength level. There is no minimum entry level of strength to use the 10/20 methodology. RPE 6 should be super light! Using your numbers, 120 pounds sounds about right.
One piece of advice as you are getting started out: keep it simple, keep the assistance work simple and don’t over think it.