[Solved] remove records which have less than 95 percentile value counts

by


I was looking to keep the 95th percentile not the percentage.
Hence the solution is:

get the frequency of each part number and add it to the dataframe

repair['FREQ'] = \
repair.groupby('PART_NO', as_index=False)['PART_NO'].transform(lambda s: s.count())
repair.head()

filter out rows of repair where repair.freq is greater than or equal to the 95th percentile: 19600

repair = repair[repair.FREQ < repair.FREQ.quantile(.95)]

solved remove records which have less than 95 percentile value counts