WASHINGTON — Justice Anthony M. Kennedy has long been troubled by extreme partisan gerrymandering, where the party in power draws voting districts to give itself a lopsided advantage in elections. But he has never found a satisfactory way to determine when voting maps are so warped by politics that they cross a constitutional line.

After spirited Supreme Court arguments on Tuesday, there was reason to think Justice Kennedy may be ready to join the court's more liberal members in a groundbreaking decision that could reshape American democracy by letting courts determine when lawmakers have gone too far.

Justice Kennedy asked skeptical questions of lawyers defending a Wisconsin legislative map that gave Republicans many more seats in the State Assembly than their statewide vote tallies would have predicted. He asked no questions of the lawyer representing the Democratic voters challenging the map.

There was something like consensus among the justices that voting maps drawn by politicians to give advantage to their parties are an unattractive feature of American democracy. But the justices appeared split about whether the court could find a standard for determining when the practice was unconstitutional.

"Gerrymandering is distasteful," said Justice Samuel A. Alito Jr., "but if we are going to impose a standard on the courts, it has to be something that's manageable."

Some of the court's more liberal members said the problem represented a crisis for democracy and that the Supreme Court should step in.

"What's really behind all of this?" Justice Ruth Bader Ginsburg asked. She answered her own question: "The precious right to vote."

In extended remarks, Chief Justice John G. Roberts Jr. expressed worry that the court's authority and legitimacy would be hurt were it to start striking down voting districts in favor of one political party or another.

"That is going to cause very serious harm to the status and integrity of the decisions of this court in the eyes of the country," he said.

The Supreme Court has never struck down an election map on the ground that it was drawn to make sure one political party wins an outsize number of seats. The court has, however, left open the possibility that some kinds of political gamesmanship in redistricting may be too extreme.

The problem, Justice Kennedy wrote in a 2004 concurrence, is that no one has devised "a workable standard" to decide when the political gerrymandering has crossed a constitutional line.

On Tuesday, he pressed Erin E. Murphy, a lawyer for Wisconsin lawmakers, about whether a state law could require drawing districts to have the maximum number of votes for a given political party. Other justices followed up on the point, and Ms. Murphy gave equivocal answers.

Justice Kennedy grew frustrated. "I have to say that I don't think you ever answered the question," he said.

Justice Sonia Sotomayor asked Ms. Murphy more fundamental questions.

"Could you tell me what the value is to democracy from political gerrymandering?" Justice Sotomayor asked. "How does that help our system of government?"

Ms. Murphy said that gerrymandering "produces values in terms of accountability that are valuable so that the people understand who isn't and who is in power."

That did not seem a sufficient reason, Justice Sotomayor said, "to stack the decks."

Much of the argument concerned various statistical tests for identifying extreme gerrymandering. Misha Tseytlin, Wisconsin's solicitor general, said the challengers were relying on flimsy and hypothetical social science evidence.

"Plaintiffs are asking this court to launch a redistricting revolution based upon their social science metrics," he said.

Chief Justice Roberts told Mr. Smith that courts are poorly equipped to evaluate social science data. "It may be simply my educational background," the chief justice said of the studies before the court, "but I can only describe it as sociological gobbledygook."

Other justices seemed more comfortable with the studies.

"This is not kind of hypothetical, airy-fairy, we guess, and then we guess again," Justice Elena Kagan said. "I mean, this is pretty scientific by this point."

Justice Sotomayor said that "every single social science metric points in the same direction."

There may be close cases, Justice Kagan said, but this was not one of them. "This map goes pretty much over every line," she said.

Justice Stephen G. Breyer, in remarks that may have been aimed at Justice Kennedy, sketched out a series of criteria that he said amounted to a workable standard. He said courts should act only when there is one-party control of the state government and a map that creates a persistent and unjustified partisan advantage that is "an extreme outlier" when compared to other maps.

"I suspect that that's manageable," Justice Breyer said.

The case, Gill v. Whitford, No. 16-1161, started when Republicans gained complete control of Wisconsin's government in 2010 for the first time at the beginning of a redistricting cycle in more than 40 years. Lawmakers promptly drew a map for the State Assembly that helped Republicans convert very close statewide vote totals into lopsided legislative majorities.

In 2012, after the redistricting, Republicans won 48.6 percent of the statewide vote for Assembly candidates but captured 60 of the Assembly's 99 seats.

Democratic voters sued, saying the maps violated the Constitution. "This is one of the most extreme gerrymanders ever drawn in living memory of the United States," Mr. Smith said on Tuesday.

The case is part of a larger debate over politics in redistricting, one that has taken on new urgency with the advent of sophisticated software. Both parties have engaged in partisan gerrymandering, but these days, Republicans have an advantage following a wave of victories in state legislatures that allowed lawmakers to draw election maps favoring their party.

Some critics, like Arnold Schwarzenegger, a Republican and the former governor of California, say districts should be drawn by independent commissions rather than politicians. Prominent Democrats, including former President Barack Obama and his first attorney general, Eric H. Holder Jr., are pushing to undo the redistricting gains Republicans made after the 2010 census when the next census is taken three years from now.

Outside the court during the arguments, several dozen activists rallied with signs that read "Equal Districts Under Law" and "Hands off our Districts!"

Mr. Schwarzenegger, who attended the argument, said afterward that he is hopeful that the justices will put a stop to partisan gerrymandering.

"We are here today to ask the Supreme Court to fix something that the politicians will never do," he said. "As Einstein said, those who created the problem will not be able to solve it."

Last year, a divided three-judge Federal District Court panel ruled that Republicans in Wisconsin had gone too far. The map, Judge Kenneth F. Ripple wrote for the majority, "was designed to make it more difficult for Democrats, compared to Republicans, to translate their votes into seats."

The decision was the first from a federal court in more than 30 years to reject a voting map as partisan gerrymandering.

Wisconsin officials say that the lopsided representation of Republicans in the State Legislature is a product of geography rather than gerrymandering. Democrats have packed themselves into cities, effectively diluting their voting power, while Republicans are more evenly distributed across most states, the brief said.

Judge Ripple acknowledged that the distribution of the population explains at least some part of the gap.

"Wisconsin's political geography, particularly the high concentration of Democratic voters in urban centers like Milwaukee and Madison, affords the Republican Party a natural, but modest, advantage in the districting process," he wrote.

But he added that partisan gerrymandering amplified that advantage.

This article on the critical issue of political gerrymandering appeared in The New York Times today.

###

October 3, 201 7