If you have a table with duplicate rows in PostgreSQL, you can use any of the following queries to return the duplicate rows.
Sample Data
Suppose we have a table with the following data:
SELECT * FROM Pets;
Result:
petid | petname | pettype -------+---------+--------- 1 | Wag | Dog 1 | Wag | Dog 2 | Scratch | Cat 3 | Tweet | Bird 4 | Bark | Dog 4 | Bark | Dog 4 | Bark | Dog
The first two rows are duplicates, and the last three rows are duplicates. That’s because all three columns contain the same values in each duplicate row.
Option 1
We can use the following query to see how many rows are duplicates:
SELECT
PetId,
PetName,
PetType,
COUNT(*) AS "Count"
FROM Pets
GROUP BY
PetId,
PetName,
PetType
ORDER BY PetId;
Result:
petid | petname | pettype | Count -------+---------+---------+------- 1 | Wag | Dog | 2 2 | Scratch | Cat | 1 3 | Tweet | Bird | 1 4 | Bark | Dog | 3
We can alternatively order it by count in descending order, so that the rows with the most duplicates appear first:
SELECT
PetId,
PetName,
PetType,
COUNT(*) AS "Count"
FROM Pets
GROUP BY
PetId,
PetName,
PetType
ORDER BY Count(*) DESC;
Result:
petid | petname | pettype | Count -------+---------+---------+------- 4 | Bark | Dog | 3 1 | Wag | Dog | 2 2 | Scratch | Cat | 1 3 | Tweet | Bird | 1
Option 2
We can use the the HAVING
clause if we only want the duplicate rows listed:
SELECT
PetId,
PetName,
PetType,
COUNT(*) AS "Count"
FROM Pets
GROUP BY
PetId,
PetName,
PetType
HAVING COUNT(*) > 1
ORDER BY PetId;
Result:
petid | petname | pettype | Count -------+---------+---------+------- 1 | Wag | Dog | 2 4 | Bark | Dog | 3
Option 3
Another option is to use Postgres’s ROW_NUMBER()
window function:
SELECT
*,
ROW_NUMBER() OVER (
PARTITION BY PetId, PetName, PetType
ORDER BY PetId, PetName, PetType
) AS Row_Number
FROM Pets;
Result:
petid | petname | pettype | row_number -------+---------+---------+------------ 1 | Wag | Dog | 1 1 | Wag | Dog | 2 2 | Scratch | Cat | 1 3 | Tweet | Bird | 1 4 | Bark | Dog | 1 4 | Bark | Dog | 2 4 | Bark | Dog | 3
The PARTITION BY
clause divides the result set produced by the FROM
clause into partitions to which the function is applied. When we specify partitions for the result set, each partition causes the numbering to start over again (i.e. the numbering will start at 1 for the first row in each partition).
Option 4
We can use the above query as a common table expression to return just the surplus rows from the matching duplicates:
WITH cte AS
(
SELECT
*,
ROW_NUMBER() OVER (
PARTITION BY PetId, PetName, PetType
ORDER BY PetId, PetName, PetType
) AS Row_Number
FROM Pets
)
SELECT * FROM cte WHERE Row_Number <> 1;
Result:
petid | petname | pettype | row_number -------+---------+---------+------------ 1 | Wag | Dog | 2 4 | Bark | Dog | 2 4 | Bark | Dog | 3