{"id":17492,"date":"2022-10-25T01:40:41","date_gmt":"2022-10-24T20:10:41","guid":{"rendered":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/"},"modified":"2022-10-25T01:40:41","modified_gmt":"2022-10-24T20:10:41","slug":"solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed","status":"publish","type":"post","link":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/","title":{"rendered":"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed]"},"content":{"rendered":"

[ad_1]
\n<\/p>\n

\n
\n
<\/div>\n
\n
\n

You can not use<\/p>\n

\n

target, predicted = pipe.fit_predict(df)<\/p>\n<\/blockquote>\n

with your defined pipeline, because the fit_predict() method can only be used, if the estimator has such a method implemented as well. Reference in documentation<\/a><\/p>\n

\n

Valid only if the final estimator implements fit_predict.<\/p>\n<\/blockquote>\n

Also, it would only return the predictions, so you can not use target,predicted =<\/code> but should use predicted =<\/code><\/p>\n

You got the error<\/p>\n

\n

ValueError: setting an array element with a sequence.<\/p>\n<\/blockquote>\n

because you are providing the StandardScaler()<\/code> a pandas.TimeSeries<\/code>.<\/p>\n

This is because with your method call pipe.fit_predict(df)<\/code> you only provide an ‘X’ and not an ‘y’ to the pipeline. This is fine for your first component of the pipeline “MakeFeatures” since it accepts an ‘X’ and returns ‘data’ and ‘y’, but in the pipeline the ‘y’ will not be used, because the ‘y’ has to be defined in the beginning of the fit_predict() call.<\/p>\n

Have a look at the documentation of the method here: https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.pipeline.Pipeline.html#sklearn.pipeline.Pipeline.fit_predict<\/a><\/p>\n

It states for the ‘y’ parameter<\/p>\n

\n

Training targets. Must fulfill label requirements for all steps of the
\npipeline.<\/p>\n<\/blockquote>\n

So that ‘y’ would be used as the ‘y’ for all parts of the pipeline, but yours is not defined, so None<\/code>.<\/p>\n

What basically happens with your current pipeline is therefore this:<\/p>\n

makeF = MakeFeatures(df, 2 , 24)\ntransformed_df = makeF.fit_transform(df)\n\nsc = StandardScaler()\nsc.fit(transformed_df)\n<\/code><\/pre>\n
and causes ValueError: setting an array element with a sequence.<\/code><\/p>\n
So I suggest you to update your code like this:<\/p>\n
import numpy as np\nimport pandas as pd\nfrom sklearn.base import BaseEstimator\nfrom sklearn.base import TransformerMixin\nfrom sklearn.pipeline import Pipeline \nfrom sklearn.preprocessing import StandardScaler \nfrom sklearn.linear_model import LinearRegression\n\nnp.random.seed(1)\n\nrows,cols = 100,1\ndata = np.random.randint(100, size = (rows,cols))\ntidx = pd.date_range('2019-01-01', periods=rows, freq='20min') \ndf = pd.DataFrame(data, columns=['num_orders'], index=tidx)\n      \nclass MakeFeatures(BaseEstimator, TransformerMixin):\n\n  def __init__(self, X, max_lag = None, rolling_mean_day = None, rolling_mean_month = None):\n      self.X = X.resample('1H').sum()\n      self.max_lag = max_lag\n      self.rolling_mean_day = rolling_mean_day\n      self.rolling_mean_month = rolling_mean_month\n          \n  def fit(self, X):\n      return self\n\n  def transform(self, X):\n      data = pd.DataFrame(index = self.X.index)\n      data['num_orders'] = self.X['num_orders']\n      data['year'] = self.X.index.year\n      data['month'] = self.X.index.month\n      data['day'] = self.X.index.day\n      data['dayofweek'] = self.X.index.dayofweek\n      \n      data['detrend'] = self.X.shift() - self.X\n      \n      if self.max_lag:\n          for lag in range(1, self.max_lag + 1):\n              data['lag_{}'.format(lag)] = data['detrend'].shift(lag)\n      if self.rolling_mean_day:\n          data['rolling_mean_24'] = data.detrend.shift().rolling(self.rolling_mean_day).mean()\n      \n      if self.rolling_mean_month:\n          data['rolling_mean_24'] = data['detrend'].shift().rolling(self.rolling_mean_month).mean()\n      \n      if data['year'].mean() == data['year'][1]:\n          data = data.drop('year', axis = 1)\n      \n      data = data.dropna()\n      \n      y = data.num_orders\n      data = data.drop('num_orders', 1)\n      \n      return data, list(y)\n\npipe = Pipeline([\n                 ('scaler', StandardScaler()),\n                ('Model' , LinearRegression())\n      ])\n\nmakeF = MakeFeatures(df, 2 , 24)\nmakeF.fit(df)\ndata,y = makeF.transform(df)\npipe.fit(data,y)  # where \u2018Target\u2019 is y - the output from the Class\n<\/code><\/pre>\n
Then you can use your pipeline to predict your data and evaluate the performance for instance with the r2_score:<\/p>\n
from sklearn.metrics import r2_score\n\npredictions = pipe.predict(data)\nr2_score(y,predictions)\n<\/code><\/pre>\n<\/p><\/div>\n
<\/div>\n<\/div>\n
            4<\/span> <\/p><\/div>\n<\/div>\n
[ad_2]<\/p>\n
solved Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] <\/p>\n","protected":false},"excerpt":{"rendered":"
[ad_1] You can not use target, predicted = pipe.fit_predict(df) with your defined pipeline, because the fit_predict() method can only be used, if the estimator has such a method implemented as well. Reference in documentation Valid only if the final estimator implements fit_predict. Also, it would only return the predictions, so you can not use target,predicted … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320],"tags":[1173,4334,4333,792,4335],"class_list":["post-17492","post","type-post","status-publish","format-standard","hentry","category-solved","tag-machine-learning","tag-pipeline","tag-python-3-7","tag-scikit-learn","tag-transformer-model"],"yoast_head":"\n[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] - JassWeb<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] - JassWeb\" \/>\n<meta property=\"og:description\" content=\"[ad_1] You can not use target, predicted = pipe.fit_predict(df) with your defined pipeline, because the fit_predict() method can only be used, if the estimator has such a method implemented as well. Reference in documentation Valid only if the final estimator implements fit_predict. Also, it would only return the predictions, so you can not use target,predicted ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\" \/>\n<meta property=\"og:site_name\" content=\"JassWeb\" \/>\n<meta property=\"article:published_time\" content=\"2022-10-24T20:10:41+00:00\" \/>\n<meta name=\"author\" content=\"Kirat\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kirat\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\"},\"author\":{\"name\":\"Kirat\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\"},\"headline\":\"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed]\",\"datePublished\":\"2022-10-24T20:10:41+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\"},\"wordCount\":277,\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"keywords\":[\"machine-learning\",\"pipeline\",\"python-3.7\",\"scikit-learn\",\"transformer-model\"],\"articleSection\":[\"Solved\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\",\"url\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\",\"name\":\"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] - JassWeb\",\"isPartOf\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#website\"},\"datePublished\":\"2022-10-24T20:10:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/jassweb.com\/solved\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed]\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/jassweb.com\/solved\/#website\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"name\":\"JassWeb\",\"description\":\"Build High-quality Websites\",\"publisher\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/jassweb.com\/solved\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/jassweb.com\/solved\/#organization\",\"name\":\"Jass Web\",\"url\":\"https:\/\/jassweb.com\/solved\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"contentUrl\":\"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png\",\"width\":693,\"height\":132,\"caption\":\"Jass Web\"},\"image\":{\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31\",\"name\":\"Kirat\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"contentUrl\":\"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586\",\"caption\":\"Kirat\"},\"sameAs\":[\"http:\/\/jassweb.com\"],\"url\":\"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] - JassWeb","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/","og_locale":"en_US","og_type":"article","og_title":"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] - JassWeb","og_description":"[ad_1] You can not use target, predicted = pipe.fit_predict(df) with your defined pipeline, because the fit_predict() method can only be used, if the estimator has such a method implemented as well. Reference in documentation Valid only if the final estimator implements fit_predict. Also, it would only return the predictions, so you can not use target,predicted ... Read more","og_url":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/","og_site_name":"JassWeb","article_published_time":"2022-10-24T20:10:41+00:00","author":"Kirat","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kirat","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/#article","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/"},"author":{"name":"Kirat","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31"},"headline":"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed]","datePublished":"2022-10-24T20:10:41+00:00","mainEntityOfPage":{"@id":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/"},"wordCount":277,"publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"keywords":["machine-learning","pipeline","python-3.7","scikit-learn","transformer-model"],"articleSection":["Solved"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/","url":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/","name":"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed] - JassWeb","isPartOf":{"@id":"https:\/\/jassweb.com\/solved\/#website"},"datePublished":"2022-10-24T20:10:41+00:00","breadcrumb":{"@id":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/jassweb.com\/solved\/solved-hello-two-questions-about-sklearn-pipeline-with-custom-transformer-for-timeseries-closed\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/jassweb.com\/solved\/"},{"@type":"ListItem","position":2,"name":"[Solved] Hello, two questions about sklearn.Pipeline with custom transformer for timeseries [closed]"}]},{"@type":"WebSite","@id":"https:\/\/jassweb.com\/solved\/#website","url":"https:\/\/jassweb.com\/solved\/","name":"JassWeb","description":"Build High-quality Websites","publisher":{"@id":"https:\/\/jassweb.com\/solved\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/jassweb.com\/solved\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/jassweb.com\/solved\/#organization","name":"Jass Web","url":"https:\/\/jassweb.com\/solved\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/","url":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","contentUrl":"https:\/\/jassweb.com\/wp-content\/uploads\/2021\/02\/jass-website-logo-1.png","width":693,"height":132,"caption":"Jass Web"},"image":{"@id":"https:\/\/jassweb.com\/solved\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/65c9c7b7958150c0dc8371fa35dd7c31","name":"Kirat","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/jassweb.com\/solved\/#\/schema\/person\/image\/","url":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","contentUrl":"https:\/\/jassweb.com\/solved\/wp-content\/litespeed\/avatar\/1261af3c9451399fa1336d28b98ea3bb.jpg?ver=1776403586","caption":"Kirat"},"sameAs":["http:\/\/jassweb.com"],"url":"https:\/\/jassweb.com\/solved\/author\/jaspritsinghghumangmail-com\/"}]}},"_links":{"self":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/17492","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/comments?post=17492"}],"version-history":[{"count":0,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/posts\/17492\/revisions"}],"wp:attachment":[{"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/media?parent=17492"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/categories?post=17492"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jassweb.com\/solved\/wp-json\/wp\/v2\/tags?post=17492"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}