我麵臨以下錯誤而擬合模型。我試圖運行一個模型的交叉驗證的管道內。下麵是數據轉換的代碼片段:
qd = QuantileDiscretizer (relativeError = 0.01, handleInvalid =“錯誤”,numBuckets = 4, inputCols =(“時間”),outputCols = [" time_qd "]) #正常化向量標量= StandardScaler () \ .setInputCol (vectorized_features) \ .setOutputCol(“特性”)#編碼器VesselTypeGroupName編碼器= StringIndexer (handleInvalid =“跳過”)\ .setInputCols([“類型”])\ .setOutputCols ([" type_enc "]) # OneHot編碼分類變量encoder1 = OneHotEncoder () \ .setInputCols ([“type_enc”、“ID1”,“ID12”、“time_qd”)) \ .setOutputCols ([“type_enc1”、“ID1_enc”,“ID12_enc”、" time_qd_enc "]) #彙編裝配變量= VectorAssembler (handleInvalid =“保持”)\ .setInputCols ([“type_enc1”、“ID1_enc”,“ID12_enc”、“time_qd_enc”)) \ .setOutputCol (“vectorized_features”)特性在一個炎熱的編碼後的總數不會超過200。模型代碼如下:
lr = LogisticRegression (featuresCol =“特性”,labelCol =“標簽”,weightCol = classWeightCol) pipeline_stages =管道(階段= [qd、編碼器、encoder1,彙編,標量,lr)) #創建邏輯回歸參數網格參數調優paramGrid_lr = (ParamGridBuilder () .addGrid (lr)。regParam,(0.01, 0.5, 2.0)) #正則化參數.addGrid (lr)。elasticNetParam,[0.0, 0.5, 1.0]) #彈性網絡參數(脊= 0).addGrid (lr。麥克斯特,[1、10、20])# .build的迭代次數())cv_lr = CrossValidator(估計量= pipeline_stages estimatorParamMaps = paramGrid_lr,評估者= BinaryClassificationEvaluator (), numFolds = 5,種子= 42)cv_lr_model = cv_lr.fit (train_df).fit方法拋出以下錯誤:
我已經嚐試增加司機內存(28 gb ram和8芯),但仍麵臨著同樣的錯誤。請幫助這個問題的原因是什麼。
