Japan will limit asylum seekers' right to work from Monday, making changes to its refugee system that are likely to swell the numbers of those in detention centers, the justice ministry said, prompting refugee groups to flag humanitarian concerns.