The ATGWU and British Midland's ground handling staff have rejected the airline's invitation to go to the Labour Court for talks.